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ABSTRACT 

In continaing research into the technology of 
training, a study was undertaken to devise guidelines for applying 
programed instruction to training courses that involve the learning 
of principles and rules for use in problem solving. As a research 
vehicle, a portion of the material in the Army's Programing 
Specialist course was programed to explore several different factors 
in using automated instruction to teach computer programing* 
Experimental versions of tbe course were administered to over 900 
subjects in various experimental groups. Criterion and retention 
tests based on actual job problems vere used to measure subject's 
performance, along vith in*training measures* Results in a series of 
proropting/conf irokation variations indicated that giving subjects 
extensive stimulus support daring training helps motivate them and 
improves scores during training, but hampers them in using vhat they 
have learned. Requiring subjects to fully vrite out rules during 
training hindered them in developing problem- solving skills applying 
these rules; hovever, using mnemonics daring training aided subjects 
in retaining vhat they had learned. (Aathor) 
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FOREWORD 



The rcdcarcli describee' in tills report was pcrfot'imxi by the flun\:in 
Resources Research OHlce under Wot^k Ifnit MI^yrUOD. Reseiirch Tot* Pi^ogramed 
Inst miction in M ilitary Tr^aining, Tfie objectives of Sub-irnit Jl oi' that Work Unit 
wer^o to develop guidelines Tor applying programed instruction to military tasks 
requiring tile learning of principle*^ and rules, especially tliose ap[>lying to 
computer programming. 

\ The vehicle Tor the research was the development of an at(tomated instt^uc- 
tioual t)acl^agc for teaching basic computer programming instructions in the 
f^Ml'^LDATA symbolic coding language. The Tina! version of the course is 
entitlcdi i^asic Computer Programming: A Self-Instructional Course (with 
Answer FiooIUct), June 1907* 

The rescat^cli was conducted at lIumRRO Division No. 1 (System Operations). 
The present Director of Research of the Division is Dr. J. Daniel Lyons i 
Dr. Ai^thur J. IToehn was Director of Research when METHOD II was initiated: 
Dr. Robert J. Seidel was Wot^k Unit Leader^ 

HumRRO staff members who contributed to the research include Dr. Iris 

C. Rotberg, in the early stages; Dr. Eugene F* MacCaslin, task analysis; 
Dr. Donald Rc>Tiolds, Dr. Harold Wagner, and Dr. Richard D* Behringer, data 
collection; Dr, Harold G. Hunter, course preparation and report preparation; and 
SP 4 Wayne S* Carpenter, course preparation. 

The superintendents of the school systems in the metropolitan Washington, 

D. C., area who provided facilities and students for the administt^ation of the 
experimental pt^ogramed instructional cout^se were: Dr. J. C. Albohm, Alexandria, 
Virginia; Dr* H* Wilson, Associate Superintendent for Instruction, Arlington 
County, Vi t^ginia; Dr. C^ F. Hansen^ Washington, D.C; Mr. E.C . F'underburk, 
FairfEix County, Virginia; Dr* H, Elseroad, IVlontgomery County, Maryland; 
and Mr. W.S. Schmidt, Prince Georges County, Maryland. 

HumRRO research for the Department of the Army is conducted under 
Contract DA 44-188-ARO-2 and Army Project 2J024701A712 01, Training, 
Motivation, Leadership Research. 



Meredith P, Crawford 
Director 

Human Resources Research Office 
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SUMMARY AND CONCLUSIONS 



Military Problem 

The military problem to which this study was addressed is primarily that of the need for 
research in the technology of training- with development of a potentially useful course serving 
both as the research vehicle and as a secondary objective. 

In recognition of the need for more efficient training of military personnel to meet the 
increasing demands for more complex knowledge and skills, research and development on auto^ 
mated instruction is continuing on □ widespread scale. While solution of training problems for a 
particular course represents an important activity- contributions that have broader application 
can be made by establishing general guidelines for developing automated instruction for a ciass 
of instructional content. Toward this end, the principal objective of METHOD II research has 
been to develop general guidelines for the application of programed instruction to tasks requiring 
the use of principles ancf ruies in problem solving. 

Increasing complexities in military systems have created a nef*d for faster and more effi- 
cient methods o( information processing and a consequent increase in the importance of training 
for computer programmers, whose work involves the application of principles and rules. The 
activity chosen for the research^ therefore^ was the development of an automated instructional 
package for a portion of the Automatic Data Processing Specialist {ADPS) programming course 
(MOS 745.1). The instructional content was concerned with the basic computer programming 
instructions nnd FIELDATA symbolic coding languagei pertinent to the MOBIDIC computer. 
Emergence of a usable military course of instruction would be a desirable by-product of 
the research* 

Research Problem 

Although programed instruction is beginning to find application in full-length operational 
courses, much of the research on the technique has been restricted to short-course laboratory 
studies, using material requiring simple memorization. Most operational trainingi on the other 
hand* requires learning complex material— "connecting" a set of stimuli (concepts) and symbolic 
"mediating" responses- in contrast to "connecting" a single stimulus and a single overt response. 
Further, for training- a distinction must be made between learning per se (i.e., developing a base 
knowledge of and ability to use what was learned to solve problems. 

To help bridge the gap between laboratory research and training- the METHOD II research 
was planned as a large-scale experimental study to explore a variety of factors in using automated 
instruction to teach fundamentals of computer programming. The study provided "real'' training 
content- an environment more like a school than a laboratory^ and complex material to be learned. 

Variables studied experimentally in presenting programed instruction were (a) degrees of 
prompting-providing correct answers to the student before he responds^ (b) degrees of coniir- 
mad 073— providing correct answersfor the student ofter he responds, (c) effects of verta/ization— 
requiring students to write or name rules, nnd (d) effects of vadety o/ practice. 

Since prompting and confirmation affect the smallest unit of a program^ the individual 
instruction frome. study of these variables deals with the fundamental aspects of preparing 
programed instruction. Verbalization which bas been used in other research indicate whether 
the student understands the nraterial, was used in this study to explore the re lati on shiy between 
the student's ability to verbalize rules and principles and his ability to actually use them in 
problem-solving performance on realistic materials. 



Approach 

The research involved four major typ**£ of cctivHies: 

(1) Literature review. The literature on programed instruction and learning principles 
was reviewed to select concepts and techniques for experimental study. 

(2) Program development. After attending military and industrial computer program- 
mer classesx researchers analyzed the programmer's work, with the aid of operational military 
personnel. Portions of the ADPS programming course were selected for expert men tat ion x and 
training content for an experimental course was developed jointly by military instructors and 
research personnel. 

(3) Experimentation. Materials for the experimental course were divided into five 
parts of increasing difficulty. Training content was translated into programed instruction formal 
and then prepared to reflect varying degrees of the experimental variables. Training experiments, 
factorial in design, were then conducted: a relatively small experiment using the first two parts 
of the course, and a larger-scale effort using all five parts of the course. 

(4) Course revision. The principal research findings were applied in deveJoping a 
revised version of the course in programed instruction format. 

The experimentation phase of the research may be summarized as follows: 

Experiment lA— The prompting/confirmation and the verbalization variables were 
studied during a three-day period of training and testing, using 60 high school students. The 
first two portions of the five-part course were the subject matter. As the prompting/confirmation 
training condition, one-half the subjects wrote answers after being given the needed information, 
and the other half wrote answers before receiving the information. Alternative versions of ver- 
balization were used; during trainingi one group of subjects was required to write the entire rule 
applicable to the computer programming problems they were solving, a second group wrote the 
appropriate mnemonics (name) for the rulei and a third group neither wrote nor named the rule 
during their instructional work. 

Experiment IB— This experiment was the first portion of the large-scale study. It 
initially involved 805 volunteer subjects from local high schools, about half of whom remained 
in the program at thp close of the IB portion of the training. It covered Parts 1 and II of the 
course and the materials used were identical to those in Experiment lA* The same prompting/ 
confirmation conditions also applied; study of the use of mnemonics was continued with groups 
that named the rule or did not name the rule in training, but the treatment requiring writing of the 
entire rule was omitted. This experiment and Experiment 2, togetheri covered 10 weeks, with 
instruction given once a week. 

Experiment 2— This experiment was the second portion of the large-scale study, fol- 
lowing Experiment IB in time and covering the more compiex content of the course {Parts HI, IV, 
and V}- The students who had completed Experiment IB continued into Experiment 2; a total oL 
345 completed the whole five-part program. The treatments included expanded variations of the 
prompting/confirmation conditions. They also included a variety- of -practice variable; during 
learning one group of subjects practiced solving problems in a single context while a second 
group was given three different variations of problems. 

For all of the experimentation, the performance tests used as final criterion measures 
(and also as retention tests given at periods of one day to four weeks after training) represented 
the kinds of computer programming problems typically encountered on the job by programmers 
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with a comparable amount and kind ol training. Work on similar practice problems within the 
course provided error rate data on learning during training. 

Following the experimental administrations, the course was revised to incorporate findings 
from the research and apply experience gained. This revision included simplifying the content 
and improving the format, The course was then administered to seven college freshmen ana six 
high school seniors, all of whom completed it. 

Results 

Experimental Effects 

(1) The results of Experiment lA showed that: 

(a) Writing out rules of computer programming during training hindered students on 
the performance tests, despite the (act that they learned to write the rules better than the 
other students. 

(b) Prompting was associated with significantly better learning than was confirmation, 
as measured by error rotes during training. However^ a reverse tendency appeared on the 
application of teaming to problem solving in the criterion performance test. 

(c) Practice in naming the rules of computer programming {rather than writing them out) 
appeared to aid criterion performance. 

(2) Experiment IB added to the reliability and general izqbllity of Experiment lA results by 
duplicating them unde: different administrative procedures. 

(3) The results v'>f Experiment 2 showed that: 

(a) The more information (either prompting or confirmation) the students received during 
training, the more likely they were to complete the course. 

(b) For those who completed the course, more prompting or confirmation reduced error 
during learning^ but increased error on the criterion tests. 

(c) Students who dropped out had performed much more poorly than they had tht?ught 
ttiey would. Students who finished the course had expected to perform only slightly better than 
they did. 

(d) Variety in practice problems led to better criterion scores than did no-variety. 

(e) Results on the final retention test four weeks later yielded the same pattern as the 
criterion tests. 

Course Effectiveness 

(1) Taking into consideration the differences in administration conditions between the Army 
school situation ond an experimental setting aimed primarily toward comparing techniques rather 
than maximizing scores, the course seemed to be effective in certain of the experimental condi- 
tions. Specifically* the average scores on criterion tests in the first two parts of the course 
were approximately 80-85% correct* Under the optimal experimental conditions in the most com- 
plex portion of the course, the scores dropped to approximately 50% correct. Average time for 
completion of the full five-part course (Experiments lA and 2) was about 27 hours. 

(2) In the pilot administration of the final revision of the course, four of the seven college 
freshmen given the course had scores of 90%> or better on the final criterion test and the lowest 
score was in the high 70s; median completion time was 26 hours. The high school students' 
scores averaged slightly lower, in the 80si with a median completion time of 31 hours. The two 



high school students who obtained the highest scores were in the bottom third ol their 
senior class. 

Conclusions 

(1) Hesutts suggest the following guidelines for preparing programed instruction for learning 
rules and principles: 

(a) A large amount of prompting and confirmation is important as o motivational device 
to keep students interested in continuing training. However, a smailer amount of prompting and 
confirmation^ while producing more errors during learning and lowering motivation^ appears to 
foster better ability to use the results of training. The advantage of le^s prompt ing/confirmat ion 
for ability to use learning seems consistent and long-lasting. 

(h) Requiring students to write out rules during their training hinders the development 
of problem-solving skills using these ?ules. 

(c) Variety in practice problems facilitates the learning of problem-solving skills. 

(d) Early training using mnemonics (naming the rules) aids later learning and retention, 
particularly as material becomes more complex. 

(2) Because C)f the nature of the course content, it would appear to be useful for either 
officer or enlisted personnel who need some introductior) to computer programming^ whether for 
administrative or supervisory activities- or for some allied operational activity. 
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Chapter! 
RATIONALE 



MILITARY PROBLIlM 

As present and future defense systems continue to increase in size and 
eoniplexity, greater demands Tor information and skills arc being placed on 
military personnel. Besides the requirements of learning new weapons systems, 
changes in current equipment and tactics impose broad, continuing requirements 
for adaptation of skills already learned. Taken together, these conditions demand 
the continuing development of methods f'or constructing shorter, more effective 
training programs. 

Much necessary training research has been di rected toward solving problems 
in specific courses of instruction. Less research has been aimedat establishing 
a body of knowledge, sets of principles, and effective guidelines for training 
development in general, or for specific categories of training content. Besides 
the need for more effective individual training programs, then, there is a need 
to develop a body of training technology. Specific findings need to be brought 
together in a systematic way, so that they may be more readily applied to future 
Army training programs, 

A category of training content that has both great importance and broad 
application is the learning of principles and rules for use in problem solving. 
It was therefore decided that the Mli]TPTOD H research would deal with guide- 
lines for applying programed instruction (PI) to Army training courses that 
involve the learning of principles and rules . 

The use of principles and rules is typified in the task of programming for 
automatic data processing systems. The programmer's Job is problem solution, 
that is, he analyzes information given and prepares a solution to the problem in 
a form acceptableto a computer. For the Army, training computer programmers 
is a matter of considerable urgency because the increasing complexity of weapons 
and operations syj^tems requires highly effective information processing possible 
only with computers. Therefore, the vehicle chosenfor METHOD II research was 
an automated instructional package for selected portions of the Automatic Data 
Processing Specialist (ADPS) (FIKLDATA) programming course. 

From work with such materials, it was hoped that the experimental findings 
of METirOD 11 would provide a beginning toward the establishment of guidelines 
for programing training content that consists of rules and principles, A secondary 
research goal would be development, as a research by-product, of cou rse material 
that could be used for military instruction, 

SCIENTIFIC [.BACKGROUND FOR THE RESEARCH 

Several facets of theory and exploration which led to the current set of 
experiments are siummari^ied in this section; 

(1) The area of programed instruction is examined from theoretical 
aspects, as well as for its practical value as a controlled environmental setting 
in which to do tntining research, 

(2) Tile role of errors in training and training research is discussed. 
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(3) The range, difficulty, cind variety of indtructioriEil content are 
coniiidei'eci in tenna of coneept foi'tncition pi-ocesses, ciii contracted with simple 
learning such rote men^orization. 

(4) The diiitinction botweenlearning concepts and usingthem in iiolving 
problemi? is given eniphurfi.^, 

Thei^e factors are considered against a backdrop of the various kinds of 
content that have been used in the laboratory setting. Traditional pcychologieal 
researeh (generally uding isolated bits of verbal material) is eontrasted with 
the kind^ of material that must be dealt with in the everyday teaching and train- 
ing situation, that is, conneeted discourse, hierarchieally ordered and related 
conceptual materiaL Research pertinent to the specifie studies undertaken in 
Work Unit METHOD is discussed in order to bring into focus the particular 
experimental hypotheses to be investigated. 

Programed Instruction: Value as Theory and Technique 

Because of its potential for applying Skinnerian experimental techniques to 
the area of education, programed instruction has been viewed as a panacea for 
problems of teachingl it is not that. On the other hand, programed instruction 
has also been termed merely another teaching aidj it is not that either. It is a 
medium of instruction that can readily provide for the application of psycholog- 
ical principles to teaching. It can also provide a testing ground for uncovering 
laws of learning, using ay a laboratory the highly controlled educational environ- 
ment that PI makes possible. 

Ironically, the advances in programed instruction to date have come not 
from applications of theoretical developments but rather from a technique 
incidental to programed instruction per se — that of task analysis, which has 
existed in the field of military and industrial psychology for many years. With 
this tool a curriculum developer who is planning instruction is directed to a 
focus on the behavioral end-products of training (or edueation)* There is no 
doubt that this analytic tool has been helpful in improving eurricula through the 
development of programed instruction; however, it provides marked improve- 
ment in conventional teaching as well* Curriculum improvement deriving from 
this technique is best conr^idered an engineering improvement, to distinguish it 
from advances that might be due to the development of learning theory. 

The advent of programed instruction has also resulted in a focus upon the 
individual student. This represents an administrative breakthrough rather than 
a development from the psychology of individual differences. For years teachers 
have been aware of the importance of a student's individual characteristics in 
instruction, but crowded classrooms and the need to serve the average" student 
have interfered with teachers* attempts to take advantage of individual eapacities. 
The teaching machine and the programed booklet have provided a distinct step 
forward in this regard. 

Educational research literature is rife with studies showing no differences 
between the so-called conventional" and the programed instructional technique 
(cf,, Schramm, T), Usually, the data indicate that one can teach the same amount 
in less time using programed instruction than using conventional instruction, or 
else the development of proper behavioral objectives for a couri^e results in 
improvement for both programed instruction and conventional teaching. 

Existing courses in programed instructional format reprej^ent constellationi^ 
of learning factors such as frequeney of reinforcement, amount of information 
per presentation unit, and activity of the student. No PI product truly represents 
a particular learning theory. The intent in the present research has been to try 



to examine sonio of tlio tr^aining factors {o,g,, amount of information given prior 
to requiring a response) \vitlioiit attempting to label the program according to 
whether it i^:^ related to one or' L^nother or some combination of PI viewpoints. 

To date, Birnplc devices {tht^ progri^nied booklet and small teaching machines) 
have been the prinuir^y vehicles for^ PL Such devices, in turn, huve been quite 
adiiptable to the techniques of operant conditioning— learning a response by beinj^ 
re\vurdod iiy a "consequence" of the response. These techniques {developed 
mostly with rats and pigeons) for increasing the likelihood of a response have 
become more or^ less the modus operandi of the curriculum developer in the 
fieldofprogramedinstruction. They include the training procedure of successive 
appt'o>cimation {with niiriinial, if any, errors), particular reinforcement schedules, 
immediate feedback, and active participation by the student. 

Cook and associates {e,g., 2) and Angell and Lumsdaine {3), and others as 
well, have found that prompting (giving the student specific information before 
asking hinito respond) is significantly more effective than confi rmation {giving 
him correct inl'ormation after he responds) for serial learning. Gagn^ and 
Brown (4), in a study dealing with conceptualmaterlal, compared several com- 
binations of prompting and confirmation {although they did not use those terms) 
in the learning of principles for generating number series. Their guided dis- 
covery {confirmation) group, medium difficulty condition, appeared tci be most 
effective, \vhile the rule and example {prompting) condition, low difficulty level, 
was least effective. 

Thus, the results obtained by Angell and Lumsdaine suggest that the subjects 
should be given substantial information before being required to respond, while 
the findings of Gagnl and Brown suggest that subjects should be required to dis- 
cover the answers themselves. The precise factor that causedthese experimental 
differences cannot be determined because of the differences in the research 
paradigms used and because of many other factors that were not comparable 
across the experiments or between conditions within an experiment. 

I^ffects of Errors on Learning 

The use of operant conditioning techniques, together with an insistence on 
a low error rate -during learning have combined to produce PI materials char- 
acterized by low difficulty. Low error rate requires easy-^to-undfirstand 
material, and the operant maxim of high student participation encourages the 
breaking of the material into small (hence, low error rate) segments, with a 
student response after each segment. 

Whether a low rate of error is desirable during learning may be questioned 
on the basis of data developed in studies dealing with level of aspiration. It has 
been found, for instance, that an individual performs to his maximum when the 
learning task i&i neither too easy {low error rate) nor too hard (high error rate). 
If the task is too easy the student refuses to set a level of aspiration, or personal 
goal, and will not perform the task, through lack of interest {inattentiveness) or 
lack of motivation. On the other hand, if the task is too difficult the t^tudent will 
refuse to try, because of frustration (Lewin, 5), 

The emphasis on low error rates is based, for the most part, on studies of 
lower organisms (rats and pigeons) and, thereby, of simple types of learning, 
frimiun learning, particularly in education, tends to be much more complex— 
primarily in term.s of concept formation and problem solving. 

The focus of the present study with respect to error rate during learning 
ha.s been to manipulate certain stimulus conditions that might produce greater 
or smaller crr'or rate in a concepttml, hierarchically ordered setting. It wa^i 



ovpocted that, itnlik<* tho crtcctii? obtained in rather simple diserimination learn- 
ing, those eonditioni3 leading to greater dirfieulty dtit^ing training {e.g., few hints) 
might well t*Ci>iilt in hettor ovei^all eriterion perioi^nianee and retention. 

Kange o f Instn iet ional Cotitc^ nt 

In simple operant learning, the organism is taught to diserlniinate, or 
respond diffei^ently to diflcu^ent stimuli-^for example, to perforni in one way in 
the presenee or A and a dilterent way in the presenee of B. In most types of 
human learning, however, the student not only must diseriminate A from 13, but 
must alHO leai^n tliat A :ind 15 eacli are to be treated as belonging to elasses of 
and Hs, Kor example, a ehild learns that the fork in his hand is but an 
exanipk- of fork.s in general, all of whleh have common properties; human 
beings tend to make sueh tnduetive leaps even if not told to. 

rcvon in supposed rote learning, subjects structure the stimuli into groups, 
as shov/n l^jy results of a number of studies in paired-assoeiate learning 
(fjattig, 6). I5attig has proposed an interference-facilitation theory of learning — 
that high intra-task interference promotes inter-task facilitation. This view is 
eonsist(>nt witli the motivational ideas expressed above. That is, the dual 
assumption that error rate should be minimal during the teaching-learning 
process and that human learning is analogous to a simple discriminatiun problem 
is open toquestion^ on the grounds of lowered motivation andof oversimplification/ 

In general terms, most human learning requires that the student develop trans- 
formational behavior with respect togivenstimuli. These transformations can be 
called mediating responses or hypotheses, which exist, as it were, in the student's 
"head" and allow him to apply his learning from one situation to another^ 

The wholesale adoption of operant conditioning principles can be viewed in 
another way. Skinner (9) has criticized the use in textbooks of materials he 
termed irrelevant— materials such as analogies and examples, which might 
confuse the student. Such materials are often included to challenge the student's 
thinlcing or to make the topic more interesting; similarly, an instructor, in 
translating a book and other course materials into analogies and examples, tries 
to make the student '^understand" rather than memorize concepts. What is being 
accomplished inthe teaching-learning process is the establishment of connections 
between a delimited set of stimulus elements and a delimited set of response 
elements. Instead of covering only narrowly defined "relevant" material, the 
instructor or textbook writer who makes use of materials such as analogies may 
teach a much more complex set of relations* 

Three classes of stimulus elements can be considered: irrelevant, correct, 
and omitted- The same three classes exist for the response, but there is also 
an additional class, the incorrect response. The outcome of training experience 
may be viewed as a set of incorrect stimulus -response (S-R) tendencies com- 
peting with correct ones. 

One prevalent approach to programed instruction limits content to only those 
stimuli that are relevant, and tries to limit responses, as much as is feasible 
only to those that are correct. Although this means that no competing response 
tendencies are established, the range of S-R coverage is considerably narrowed. 
In other words, a limited sample of appropriate elements becomes connected 
b:it tho coverage is incomplete in terms of the total stimulus and response 
populations* The problem of defining what is " relevant" in instruction, to develop 
the appropriate concepts and to allow the person to discriminate ''correct" from 

Mn fiirl. rrncurrli uitfi ciMWlri tr'^d (lLsL'(nirs(: has alrrady prinlurt*!! (lata wlliili r^Mso a <]iirsliim as U^ ihc 
univf^rsai apjilit abilily al mirjurial tiiTor rale {Ma^tf .SyxlOms lnL*i 7; S(H(I(^I tunJ Itolbrrg, 8). 
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"incorrect, " is ii cvuclal tlicoretiCLil antl priicticul pr^ohlcrn Wm' pro^^iMiiu'd i r},^.! ruc- 
tion, especially (ov that ciciiiing with problcm-solviiijti; l>eluivit)i > 

Once 10 considct's ii Inroad range ol" instnictiotial content, and (.U^p:iris ri'ojn 
operant conditioning concepts, new rcsearcli qiic>Stions Linfold. One siieli qiK^j^tian 
derives from Harlow's LMM7)r* factor tlieat\y (10)> In Icarnlui^ srl r'rsiMrch, 
Harlow (11) inferred tlmt the inet^cascd conceptual ]^roficii'rK\v rfhown liy suhjcuts 
stcrnnied fi^om eliminating cr^roneous hyjDOtheses diii'ing trainin;^. His conduston, 
and the essence of his theory, suggests that encoui^aging certain kinds of erroi^s 
{i.e., iucorreet S-Rs) during learning could result in more errccti\'e elimination 
of erM^oncous hypotheses^ 

Thuij wlule eoneepts arc being formed (as is the case in niosl edui^ation), 
variety of content prol^alDly facilitates the elimination of eia^oncous hypoUie.ses 
during training. It also allows the individual Jiitudcnt to abstract the apjir^opi^iutc 
mediating skills and responses more effectively than he could in a lioniogeneous 
context. The individual can then use these mediatin*! slcills and responses to 
gcnerali^ie to other stimulus contexts after training. 

Know ing and Using: Levels of Learning 

The distinction between rote and problem-solving leai'ning is made as 
follows. Rote learning involves connecting specific stimuli to specific responses 
during training, and the test of success in training is simply whether tlie connec- 
tions have been made. Problem -solving learning, on the other* liand> involves 
training the student to abstract a common (identity) response to a class of 
stimuli; while a test of success in such training consists of knowing (tjoin^? afjle 
to nanae, state, or describe a concept), the main test rests in being able to use 
the concept effectively in solving problems (ef>, Kcndler, 12, and Gagne, 13), 

A more detailed analysis of problem-solvinglearning heightens the difference 
between it and rote learning. For problem -solving learning, distinction is usefully 
made between (a) responding during training, (b) abstracting concepts from 
responses, (c) knowing a concept, (d) being able to use the concept. 

Making this distinction is not new. For years, Gestalt psyelu^logists have 
delivered polemics (and included some demonstrations of principle learning) on 
the distinction between rote and conceptual lear^ning. The thrust of Katona's 
work on learning number series is another example. However, f>nly r^ecently 
has a study by Smith, Jones, and Tliomas (14) clearly scpar'jtted the (avo types 
of responding. Their experiment directly compared rote and ctmcci^tuiil learn- 
ing where number of stimuli per response was varied from one in j\nu\ (liote 
learning involved random grouping of stimuli, vliereas eun<.:eiif^rH nr[' nu/nlion 
included a meaningful link among stinauli within each grou|.)in^.) fft- i. ihl' llu* 
num>ber of stimuli per response improved concept learning bul - : ^ tr* 

learning, thus showing two distinct learning processes. 

Added to the identity response in most human Jeat^nint! ;m ihi 
solving responses; that is, the identity response is used in deveit^fnrr '^ ejits 
to put eoneepts together in order to perform some task, WiJl th' upi i r ^ ! i^'ch- 
niques for teaching concept formation per se {identity res[)oudin^) Ix- ihc .-.ame 
as those for teaching problem solving? Gagne(T3, p, 312)sug^esls that they may 
indeed be different forms {or at least levels) of learning. C.'^jicenl K':n'nlng 
consists of "establishment of mediating response to stimuH whirls ! i:i'r fix^n 
eacl\ other physically ( 'classifying' )<" Problem solvin^^s he feels, I.l t iir ' ('si:thlish- 
ment of a process which 'combines'two or more previously lear ned ruin's in a 
'higher-order rule\" A study by GagncT and Smitli{15) suj^gerfts that tfie verbal- 
ization of principles takes on importance as the prohleni ljeL:omcs nuire ^;amplux. 
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The distinction between concept*^ and problem solving can be illustrated in 
electronics troubleshooting* The student must understand the concept of a 
schematic diagram which represents the wlresj connectors, and other parts* 
He must know the color coding system for resistors and th^ system of naming 
pins on electronic vacuum tubes, as well as how one reads a multimeter. Super- 
imposed on all or these is another requirement: the student must determine the 
likelihood thatfoUowinga given path of potential defects will be successful in 
troubleshooting the overall system* The effects of each factor may be taught* 
So might the mechanical responses or a total task in isolation* Given the learn- 
ing and isolated practice, the student may be asked to actually troubleshoot a 
malfunctioning piece of equipment; this activity comprises applying the complex 
of concepts to solve a problem * 

Viewing the troubleshooting illustration from a theoretical point of view 
provides a more technical analysis* The student might learn (from making 
certain kinds of controlled errors during training) to recognize the effects of 
various kinds of errors in the system, and to abstract from his training 
experience an understanding (conceptual set of responses) for the task* Having 
learned the proper identity responses (concepts), the student must put these 
together into a strategy to perform the task of troubleshooting successfully 
(the problem-solving aspect of the process)* Here, it seems, is a meaningful 
distinction between understanding (concepts) of a class of stimuli and their 
use (problem solving) in a series of acts required to turn out a utjeful solution*^ 

RESEARCH OBJECTIVES 

Background for the Experimental Hypotheses 

Unfortunately, much of the research that has been done concerningthe effects 
of learning factors in programed instruction has been conducted with paired 
associates or serial learning* Little has been done where experimental treat- 
ments have been manipulated for hierarchically organized conceptualdiscourse,^ 
Yet this type of discourse— meaningful course content— is precisely the medium 
for mostj if not all, programed instructional applications* Paired-associate 
experiments generally require the subject to make a series of rote responses, 
and his learning is measured by his ability to give back these rote responses. 
Conceptual learning, however, requires the subject to abstract mediating 
responses, and learning is measured by his ability to apply these mediating 
skills to a variety of different situations* 

As indicated earlier, the present study was undertaken to examine some 
of the psychological factors in programed instruction* The review of previous 
research led to selection of the following factors for exploration: 

First, the value of a rule-giving or naming type of verbalization 
response was investigated* This factor deals with the distinction between rote 
learning and concept formation, and was intended to explore the efficacy of 
repeating or naming concepts in training, for the purpose of applying learning 
to problem -solving situations. Many current programed instruction programs 
are parallel to paired-associate learning situations in that they seem to empha- 
size reproducing material presented* Yet the program developers seem to be 
interested in teaching concepts {mediating skills)* Does verbalizing a rule aid 
or hinder learning to use the rule in problem solving? 

^This view i>i suoh £i teitohinp problem bus recently beeo implfcitly expressed by Cronbach (16) io nn 
ojccellent review of discovery exprrimrnt**- 

'A rrcpnt sludy by Ifickoy an<i Nevvtnn (17) i*; a wclt*micd cjUcmpL, f(ir ;tn (*?(f4:llent disLKssiotj 

t^f prnblcms in doinf^ PI rc^eitrth nn otitjtepVUiii nu^U»'ii^h Ttuub 
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Second, the pr^inciplc of minimal crrot* rate as tippliccl to learning 
complex, meaningful materials was questioned, L^cvel of aspiration stiidiet; have 
shown that a cet^tain amount of dUTiciiUy (e.g*, crrot^ duiang learning will lead 
to maximum motivation {i\nd iniplicitly, hi/^h scovcs on a criterion test)* 

Third, tiie impor^tancc of vat^ieticrf of o^amplos in siicli human learning 
was examined to see vvlicther' this principle generalized from simpler concept 
(learning set) studies, in which variety of examples has been found to foster 
concept formation. It would be expected to aid subjects to generalize from 
training content to operational application. 

l^ourtli, attention was given to another pair of factors of interest in pro- 
gramed instruction reseat^ch— "prompting" (providing the student specific infor- 
mation prior to asking him to respond) and "confirmation" (giving him correct 
information after he responds). For a meaningful comparison of prompting and 
confirmation, the sole element manipulated should be the amount of information 
given to the subject prior to requiring him to respond; whatever is not given to 
him then should be given after he responds to make the total information each 
subject receives comparable for the confirmation and prompting conditions. 

Experimental Treatments 

In summary, the background for the METHOD II research rested on the 
need to study, using meaningful course content, those factors that seem most 
relevant to learning how to use rules and principles. The first experiment 
(lA) examined the influence of prompting vs. confirmation, and of alternate ways 
of verbalizing rules* The results of this experiment were taken into considera- 
tion in further exploration of the same variables in the second experiment (IB). 
In the third experiment (2) other aspects of these variables were explored, and, 
through work with a variety of practice problems, the development of connec- 
tions between conceptual stimuli and mediating responses was considered* 

With the training content (writing a computer program) put in the form of 
programed instruction, e:cperiments were conducted with the following factors: 

(1) Amount of info r mation to be given to trainees prior to requiring 
responses (degrees of prompting vs. degrees of confirmation) (Experiments 
lA, in, and 2). This factor was judged especially valuable to study since its 
influence is felt in the smallest unit of a program; namely, the frame itself. 

(2) Value of written verbalization of rules during training (Experiments 
1 A and IB). Verbalization of rules has been generally used in construction of pro- 
grams to indicate understanding by the student of the material inthe program. In 
this study, interest lay in the relationship between the student's ability to verbal- 
ize rules and principles and his use of them in problem-solving performance* 

(3) Value of variety of context for practice problems during the train- 
ingon problem solving (Experiment 2). The complexity in mostlearnlngdemanded 
consideration of broad connections between conceptual stimuli and mediating 
responses. Variety of verbal contexts for the practical problems given during 
training provides the setting for establishing broad connections. Minimal varia- 
tion of problem context should encourage only simple connections. 

(4) The effectij on a prob l em-solving criterion of different amounts of 
error during training, examined throughout the series of experiments. 

As the fmal step in the research effort, an attempt wa^s made to incorporate 
major findings from the experimental work into a revised self-instructional 
course in ba.sic computer programming. The desired by-product was a usable 
military course of instruction. 
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chapter 2 



RESEARCH PROCEDURE 

APPROACH 

The major research activities— a literature review> program development^ 
experimentation, and course revision— may be summarized as follows: 

(1) The scientific literature on learning principles and programed 
instruction was reviewed as described in the preceding chapter, to provide a 
basis for selecting concepts and techniques for experimental study. Selection 
was based on importance for learning in a military training context, feasibility 
of manipulation for study, and potential for the advancement of training technology. 

(2) As the first step in developing the instructional program, the 
researchers attended computer programmer classes at the U.S. Army Signal 
School, Fort Monmouth, New Jersey and at IBM, They then interviewed super- 
visory and computer instructor personnel, systems analysts, and computer 
programmers at Fort f-Iuachuca, New Mexico, The nature of the tasks performed 
by the programmer was analyzed with the assistance of operationalprogrammers. 
Portions of the Automatic Data Processing Specialist (MOS 745,1)' programming 
course were selected for experimentation. Final content was developed jointly 
by instructors from the Signal School and research personnel. 

(3) The materials selected were divided into five parts of increasing 
complexity. Training content was translated into programed instruction format 
and then prepared to reflect varying degrees of the experimental variables. 
Three experiments were conducted; their major characteristics are summarized 
below and in Figure 1. In all three experiments, the final criterion and retention 
measures were performance tests representing the types of problems typically 
encountered on the job by computer programmers with a comparable amount 
and kind of training* The experiments were: 

Experiment lA — The prompting/confirmation and verbalization 
variables were first studied during a three-day training and test period in 1962, 
using 60 students from a local highschool as subjects* Parts I and II of the five- 
part course were used as subject matter. 

Experiment IB — This experiment was the first part of a larger study 
conducted between September 1064 and January 1965, and initially involved 805 stu- 
dent volunteers from local high schools. The variables were the same prompting/ 
confirmation conditions and part of the verbalization condition used in Experiment 
lA, and the materials were also the same. Instruction was given once a week 
during the first part of a 10-week time frame which included Experiment 2. 

Experiment 2 — Thisportion of the larger study followed Experiment 
IB in time and was applied in weekly instruction to the more complex course 
content of Parts III, IV, and V, Students who had completed the instruction in 
Experiment IB were the subjects. The prompting/confirmation variable was 
expanded and a variety-of-practice variable was introduced* 

*Un(ier ihc current MOS structure, thtj ADPS Progrnming Speciiilisl is MOS 
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Figure 1 

(4) A final version of the course in programed instruction format was 
developed by applying the principal research findings, and a pilot test was con- 
ducted, using 13 students as subjects. 



OVERALL DESCRIPTION OF TUB PROGRAMED COURSE 

The experimental programed course was directed toward the level of high 
school seniors and First-year college students. This level was chosen because 
the content material was originally drawn from portions of the Army training 
course for the ADPS Programing Specialist, at the U,S, Army Signal School, in 
which the background of the Army trainees corresponds to a high school education. 

The aim of the experimental course was to provide understanding of funda- 
mental computer programming concepts and, more important, to produce pro- 
ficiency in writing elementary computer programs. From the very beginning 
of the course, its entire conte^ct was oriented toward the writing of computer 
programs* That is, a concept was introduced, then a problem incorporating 
the concept was presented, and the student programmed the problem. The 
practice problemj5 and the problems used in the criterion and retention tests 
were chosen Tt^ojii actual ADP Job situations. 



The repertoire of primary programming commands and the symbolie 
language in whieh the eomputer programs wetx written were ehosen from 
material whieli had been used by the Army to teaeh traineeii to program the 
MOBIDIC eomptrter. The language is the FIELDATA Symbolie Language wliieh, 
although representing a .set of symbols inter-pretable by the MOBIDIC, is 
suffieiently general in form that it largely applies to almost any other eompnter 
and to the English language (the addresses). 

As nearly as possible, the goals of all subjeets were made eomparable by 
orienting them with an overview of the eoneepts of eomputer programming in an 
Introduetion to the eourse. Speeifie attention was paid to the concepts of com- 
mand, address, the accumulator, memory locations, input, and output. Also 
included were a brief description of how a computer functions, and some sample 
mnemonics as they are used in computer commands. The coverage of these 
topics was interspersed with periodic review and with questions which students 
answered to themselves. The student was instructed to continue reviewing the 
material preceding each of the sets of questions until he could answer the 
material to his own satisfaction. As the last item in the introductory material, 
the student worked through a short sample program. 

The five parts of the experimental course proceeded from the simple to the 
complex* Parts I and II dealt with primary programming; the student was intro- 
duced to storage locations, the concept of a central work area, the movement of 
numbers from place to place, and simple arithmetic operations, along with an 
introduction to elements of a symbolic data language. Part HI covered basic 
looping concepts, including general transfer commands, counting the loops, 
leaving the loops, and program preparation. Parts IV and V dealt with data 
processing, covering address modification and address arithmetic (effective 
addressing), sorting and counting data into various categories, and multiple 
address modification* 

The programed instruction materials were allprepared in the form of linear 
programs so that a student moved step-by-step straight through the course* It 
would not be correct to characterize the steps as frames, however, because the 
steps — rather than single sentences or two sentences at a time— constituted con- 
ceptual or functional units. Thus, there could have been as many as two or three 
paragraphs given to the student prior to questioning him on the content. 

Details on the course materials and the variations for the experimental 
treatments are presented in the descriptions of the individual experiments. 
Content and presentation of the course are illustrated in samples in 
Appendix A. 

The criterion tests at the end of each part of the course consisted of actual 
computer programming problems submitted by a programmer group from Fort 
Huachuca and by programming instructors from the SignalSchooL Where neces- 
sary, problems were simplified by removing aspects that would require tech- 
niques beyond the scope of the course. 

EXPERIMENT lA 
Design 

A 2x3 factorial design was used in Experiment lA (illustrated schematically 
in Figure 2). Six independent groups of subjects in a PI environment learned to 
write increasingly complex computer programs (CPs). One-third of the subjects 
periodically wrote out the content of the rules used to guide the writing of com- 
puter programs (Rules Group), one-third periodically wrote down the names of 
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the rulesi (Naming Group), and one-third wrote CPs without any verbalization 
of the rules (CP-Only Group)* Also during learning, ona-half of the subjects 
were required to write their answers after being given explicit information 
needed in the response (prompting), and one-half were required to write their 
answers prior to receiving the information (confirmatioi)* During learning, 
error rates were obtained on CP-writing for all groups and on rules and naming 
for the appropriate groups. 

Treatments co,Taposed of various combinations of the training conditions 
were administered during the PI training, and their effects were evaluated on 
criterion and retention tests given at the end of each of the two parts of 
the instruction* 

Subjects 

The subjects were 60 volunteer high school students (juniors; and seniors), who 
had been informed that they were to be paid between $1*50 and $2.00 per hour, 
depending upon the proficiency they exhibited in learning. They were assigned 
at random to one of the six training conditions. A combination of scores from 
the verbal and clerical tests of the Army Classification Battery {ACB) was used 
to measure Intelligence level for the detailed evaluation of results.' 

Training Materials 

The training materials used in this experiment were Parts I and II, covering 
an introduction to symbolic language for computer programming and primary 
programming commands of a business -type digital computer. The learning task 
was writing computer programs of increasing complexity, with the additional 
requirements imposed by the experimental treatments. The treatments were 
applied as each new concept was introduced along with the attendant requirement 
of writing a computer program, 

A different programed instructional booklet was used by each of the six 
groups of subjects (Figure 2), in accordance with the training conditions assigned 
to that group. The booklets used by the Rules and Naming (or mnemonics) Groups 

^'i'ho inlrllif^entt' tfn^asur*; wan the s;xme iis thul U[ipli(*(i by thn Army for <inlriince tnln the ADPS prr>- 
^ranimin^ course. 
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conyi^^tf^tl of ?JCJ pcLguH, including the criterion tests; booklets used by tlic CP- 
Only Ciroups had hi\ii that number oT pages. Tlic difl^urcncc in treatment bDtwc(?n 
the proinptinf^ :ind confir^nuition conditions did not afreet the *^ize of tlie booklets. 

For all groups, the firslf st^^p um.s the presentation of a concept. The ncKt 
step varied aceoixling to the expt;r*irtK*ntal tr^catment. 'L'fie following sample 
rule ean he used to illustr'ate tlie variouhi treatments: 

The CLA SOCSl^^C instr^uction clears any material 
already in the aecuniiilator and eopios the number 
from location SOCSI^C into it. 
After tiic concept tiad been presented, the portion underlined above was given to 
tlie subjt^cttt in the Kules Groups, wtio were to fill in the rest; the portion not 
Linder^lined was given to tlie subjects in the Naming Groups and they filled in tJic 
part Linderlijied. Both groups were then given a problem illustrating the concept 
and they wr^otc a computer program on the problem. The CP-Only Group went 
dir^octly from tlic presentation of the concept to the statement of the problcra. 

The pr*oni|>ting/ confirmation variable was added to the al>ove treatments by 
having subjects work on only one page at a time. The Pi^mpting Groups saw the 
entire &^(*ntf*nce first; the subjects in those gmups turned the page and completed 
the blank before working on the computer programming problem. T^^e subjects 
in the Confirmation Groups completed the blank, then turned tlie page and viewed 
tnc entire sentence after working on the problem. 

The booklets included criterion tests at the end of each part of the course. 
I^s a secondary part of the criterion tests, the ability of all subjects to write 
t. e rules and the names of the rules used in writing the CPs was measured. 

P rocedures 

On the first day students were given Part I, the section concerned with pri- 
mary programming commandSi and a criterion test for that material. On the 
second day they were first given a retention test (for Part I) consisting of four 
problems to be programmed, which aU subjects finished within 30 minutes. Imme- 
diately following the retention test> they were given Part II of the course, on 
more complex computer programs, and the criterion test for that part. On the 
third day, they were given Retention Test II, consisting of four programming prob- 
lems covering all the course material, followed by the ACB intelligence measure. 

The students were allowed a ma:dmum of four hours on each of the two days 
of training, but only one or two students required that much time on either day. 
Each student's booklets were checked at the end of his working day to ensure 
that he was actually \vorking with the material- All students appeared highly 
motivated to perform; many stayed after hours to ask further questions about the 
course work. (No substantive questions were answered in order not to contam- 
inate the e.xperimental treatments.) 

EXPERIMENTS 115 AND 2 

Design 

Taken together. Experiments IB and 2 covered c^H five parts into which the 
courtie was divided. The first two parts (used in Expei^imcnt IB) were identical 
in content to that of Experiment lA; they provided subjects with the background 
necessary for the final three parts (Experiment 2), which dealt with looping 
principle^ . I^^acli of the five pur^ts Iiad its own criterion ie^L 
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AU vLUMablus in tUc two axpetimcnts wer^o iidrninLStored factoriiilly during 
training, lilxptirhnont 115 involved the same conditions as l^xpcritnent lA, except 
that tl\c Rales dr^oap was onuttCLL That is, in Experiment IB comparisons were 
made between (a) prompting and oonfh'mation; and (b) writing the names of rules 
an<l wr^lting computer pro^^ri.ims, and riimply writing tl\e computor progi'ams.' 

After completing Par^t II of J^'xperimcnt IB, students were reassigned 
ran:lomly to treatment groups for* liiXperiment 2, which included three variables: 

(1) 0, 50, or 100% confirmation, 

(2) 0, 33, 100%, or progressive (G7, 33, 0) prompting. 

(3) No-variety vri. variety conditions on practice problems. 

The No-Variety C^roap worked on practice pi-oblems having a verbal context of 
warehouse inventory for electronic vacuum tubes. The Variety Group> however, 
had tlir^ee types of verbal contexts, the one on vacuum tubes just noted, and on 
military pay, and uniform selection. 



Design for Experiment 2 
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The design of Experiment 2 is an 
incomplete threc-faetor study, con:i- 
posed of 29 cells {see Figure 3).^ 
The practice problems were given in 
sets of three instead of singly. The 
eonfi rmation variable was admin- 
istered on a more molar level than 
that of the prompting variable, being 
applied across sets ofproblems; that 
is, either the entire set of three 
problems received confirmation or it 
did not, depending upon the confir- 
n:iation applied. Both the prompting 
conditions and the variety vs. no- 
variety treatments were applied within 
the sets of three problems. Further- 
more, the progressive prompting 
condition was administered as a 
diminishing prompt within each set 

of three problems; thus, the first problem received two-thirds prompting, the 
second one-third, and the third no prompting, A sample application of treat- 
ments is shown in Appendix A. 

In addition, an attempt was made to obtain data on the relationship between 
the stage of training at which mnemonics are introduced and the improven:ient 
in criterion performance. This was done by continuing the u^e of mnemonics 
through the course for three additional groups of students.^ 



0 33 Progressive 67 
Percent of Prompting 

Addfid VcrrbaliiariOrt tr^aimfintf i" th* 5am«^ Vgrhcrtv condition: 
Noming J^'JO^ Prompting ond 0 Confi'mot toi^ 
^0 Promptiftg and 100% Confirmotion 
No^NofTitng - lOO*e, Prompting and 0 Conftrmahon 

Figure 3 
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Groups oV 2t) sul)jects or multiples thereof worked in a single experimental 
room \vitli each student of a yet of 29 in a different experimental treatment from 
those of hU t:kvssmates. This design effectively eliminiUed experimenter dif- 
forencei:; and socioeconomic var^iahility between school sytatems, and provided 
genert^liy equivalent levels acrobti cells. An n cheek on ability levels, the CL 
portion of the Army Classification Battery was administered to the students.^ 

Subjects 

The original plan was to use Army subjects at the U*S. Army Signal School 
iov Experiments IB and 2, Because of difficulties in obtaining an adequate 
sample o\ Army subjects there, however, it vas necessary to use a civilian 
popLtlation for the experimentation. Subjects were male volunteers recruited, 
without compensation, from the junior and senior grades of high schools in 
the metropolitan Washington, D,C-, area. The schools participating represented 
all local area school systems, thus insuring a sizable range in fiocioeconomic 
status of the pupils attending. No &'ystematLc attempt was made, however, to 
take stratified samples either within or across schools on the basis of socio- 
economic status* The school systems, and the number of participating schools 
per system, were: Alexandria, 1; Arlington, 1; District of Columbia, 3; Fairfax 
County, 2; Falls Church, 1; Montgomery County, 4; and Prince Georges County, 2. 

The number of volunteer subjects beginning the course was 805; the drop- 
out rate was high (57%), with the number of subjects who completed the entire 
course being 345. Approximately 75% of the attrition occurred pi lor to criterion 
testing for Fart III, 21% prior to testing for Part IV, and 4% during Part V, The 
analyses of the experimental effects, reported in the next chapter, were made on 
the basis of the subjects who completed the entire course. 

Training Materials 

Parts I and H were contained in one booklet, the content of which was 
identical to that used in Experiment lA, Parts III, IV, and V were also contained 
in a single booklet (136 pages). Part III introduced the technique of looping. 
Part IV dealt with address modification, and Part V had to do with address 
arithmetic (a more advanced technique of addressing). 

The performance criterion tests were administered after the students com- 
pleted each part of the program, A similar test was administered four weeks 
after the course was completed, as a measure of retention. Upon completion of 
each criterion test, the students were given a Level-of-Aspiration (LOA) Scale, 
divided into intervals of 5% (see Appendix B)- They were asked to make two 
judgments; (a) how well they thought they had performed on the criterion test 
they had just taken, and (b) how well they thought they would perform on the 
next criterion test* These scales were used in computing discrepancy scores, 
obtained by subtracting the student's; actual performance on the next test from 
his l-'OA or expected score. Such scores are traditionally used to obtain an 
indication of the degree of realism with which a student progresses through a 
course; for example, a small positive discrepancy score (expectation slightly 
above performance) exemplifies the realistie, positively adjusted student. 

Proce du re's 

Students were trained in a classroom at their own or a nearby high school 
after school hours (afternoon or evening sessions) or on Saturday, They attended 

\No M]\i tosl fluui were aviiiffibJo for roupfily onc-fiflh of Llio suKjccls. 
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one three-hour sci^ijion a wt^ck. It should be noted tlu^t so long a time between 
sessions not intcnclccUvhcn the ocpcrinicntalcot^rse \vas constructed because 
the delay allows nuyrc time (or forgettingn Daily sessions would be the most 
desit^ible scliedulc Tor Lho solf-instruetional course. 

l-falhvay thtough the three-hour session the students were given ti 10 to 15 
minute bc'oak* Scats wei^e spaced as far apart :is room size would allow. The 
ratio of proctors to the students was appro>:imately 1:15. The proctory insuix^d 
that the students worked independently and tried to prevent review of materials. 

Tt^octors were instt^ucted to assist the students only with procedural or 
"structural" questions. (A **sttHictural" question would be one about a suspected 
misprhit or, in rare cases, a requetst to define an unfamiliar Army term su^h 
as KP.) Proctors did not answer substantive questions, those pertaining to 
undei^standing the content being taught. In response to those questions, the proc- 
tors yiniply told the students to do the best they could. 

l^cfore the course began, the students were given a brief outline along with 
a cautionary statement t^egarding the need for independent work and regular 
attendance. Students were informed that they would be given a final test, cover- 
ing the complete experimental course, approximately four weeks after they 
completed the course, and that they would receive certifications of completion 
aftet^ the final test. 
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Chopter 3 



RESULTS 



Taken together, the three experiments generated a staggering amOLmt of 
data. To provide a framework for viewing the results* without presenting an 
overwhelming amount of detail, it may be well to review the basicquestions that 
were being a^iked. These que^itions took the form, "What is the effectof X on Y?" 
Foj* example, "What is the effect of naming rules daring training on ability to 
solve criterion problems?" The "Xa" and "Ys" are listed below, with an indica- 
tion in parentheses of the experiment in which they were used; 



Effects of . . . 

Level of prompting (all) 
Level of confirmation (all) 
Verbalization in training; 

Writing rules (lA) 

Naming rules (lA and IB) 
Variety of practice (2) 
Intelligence (all) 
Combinations of the above 



As Measured by . . . 

Errors during training (all) 
Errors on criterion tests: 

Problem solving (all) 

Writing rules (lA) 

Naming rules (lA and IB) 
Errors on retention tests (all) 
Time to complete (lA) 
Level of aspiration (IB and 2) 
Combinations of the above 



Results are presented separately for each experiment. After the evidence 
of learning has been presented* findings relative to the various treatments (left 
hand column, above) and the effects of each treatment on the various measures 
(right-hand column, above) are discussed/ 



EXPERIMENT lA 



Treatments studiedwere prompting/confirmation andverbaliisation. Instruc- 
tion was divided into two parts (1 and II). After instruction, proficiency measures 
were taken in the form of criterion tests (I and 11) and retention tests (I and II). 
Du ring instruction^ measures were taken in each phase in the form of errors 
during training and time to complete* 

E vidence for Learning . On the criterion test after Part 1, nearly two- 
thirds of the students scored 80% or more correct, and more than a third 
scored 90% or higher. Results on Criterion Test II were comparable* Mean 
criterion and retention test scores are given in Table 1, 

Time to Complete . For both Parts I and II, there were no significant dif- 
ferences between the prompting and confirmation treatments as to the length of 
time subjects took to complete the course* The three Verbalization Groups did 
differ significantly, however, on both parts. In Part I, the Rules Group was 
slower than the other two verbalization groups. In Part IL the Naming Group 

^Unless otherwise indicated, the sloUsticnJ test used wrts the AnnJysfs o( Vnriance. 

IB 



Table 1 



Mean Criterion and Retention Test Scores by Treatments: Experiment lA 
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TabI e 2 

Mean Learning Times (or Verbalization 
Variable: Experiment lA 
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was faster than the other two groups* 
The mean learning times for the 
verbalisation conditions are shown 
in Table 2. 

As can be noted in Tables 1 
and 2, achievement was positively 
related to speed of learning. To pro- 
vide an indication of the efficiency of 
each treatment, a time-weighted score 
for each individual was computed by 
dividing the student*s criterion {or 
retention) test score by his time to 
complete the course* These scores, 
which reflected the amount learned 

pe"^ unit of time, were used in the subsequent analyses of data for the vari- 
ous treatments. 

Prompting and Confirmation Effects . During learning* as measured by 
error rates In computer- programming, the Prompting Group scored significantly 
higher than the Confirmation Group* However, the groups did not differ signifi- 
cantly on the criterion tests {including the tests on writing and naming rules) or 
the retention tests. If anything, the Confirmation Group was superior, as shown 
in Table 3* 

Verbalization Effects^ . Students required to write out the rules of computer 
programming during training did not do as well on the criterion tests as students , 
in the other verbalization groups. The Rules Group also had the highest spread 
of scores on the criterion tests, as shown in Table 1* (Later supplementary 
analyses indicated that this effect was particularly evident in students of 
lower intelligence.) 
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Table 3 

Mean Time- Weighted Criterion and Retentian Test Scores 
for Prompting/Confirmation Variablei E?tperiment 1A^" 
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The mean score of the Rules Group on tests of computer programming 
(Criterion and Retention Tests 1 and II) was consistently lower than the mean 
of the Naming Group or the CP-Only Group (Table 4). Two-group comparisons 
(19) indicated that this difference was statistically significant in all four cases. 

Toble 4 



Mean Time^Weighted Criterion and Retention Test Scores 
for Verbalization Variable: Experiment lA" 
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On the criterion and retention tests following Part I, the CP-Only Group 
made the highest scores, while on the tests after Part II the Naming Group 
seemed to be superior. However, this tendency was not statistically significant. 

The inferior scores made by the subjects in the Rules Group in solving 
criterion problems did not seem to be due to poorer learning per se , since they 
scored higher on those criterion items specifically related to their training, 
namely, the writing out of rules. In Part I this difference in performance was 
not significant, but in Part II it was (p < .05). On the other hand, the Naming 
Group answered the naming questions on thp Part II criterion test significantly 
better than the other two groups (p<.05), and also performed well on tests of 
computer programming* Thus, it appeared that both groups learned the addi- 
tional materials presented during training, but that the subjects in the Rules 
Group were unable to apply their learning to problem solving/ 

Intelligence . Intelligence, as measured by scores on parts of the Army 
Classification Battery, was significantly and positively related to criterion 

V'riterion nieosures of problem srJving Kcre luglily rclinhle by lloyt^s'J'est (20)- CfMtffit ientH mnpcd from 
79 to 89 (p < 01), 
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soorc\s In liU ccises. No iator-Liction.s iippecu^cd Ix^twcen intcili^tMu r /. i^t - - 
criterion soorcs rcUitcd to tliu cxpCM'hnontLil vurunl^lerf, 

i:xpi-;rimi-:nt in 

The second experiment was liasically ii vcpirM of tlie i'ir.sl. )ts priniiuy 
Tunctions were to conrirm the earlier Hndings and proviilo the introductory 
instruction necessary for the mot^e complex materials presented in I'^xpeinment 2. 

Varia[)les stt^died in Experiment IB were the same as those for ]A except 
that the Kules Oroiip was deleted from the verbalization variable. It was 'ntended 
that the same jTieasures woLiki aLso be taken, with the addition of u Level-or- 
Aspiration measure. Flowever, difficulties attending the adnunistt\ition of a 
large study over several months, using vohmteers from many different school 
systems, resulted in the loss of the time scores, and a portion (20%) of the 
intelligonee scores. The high dropout rate (57% for ICxperinients IB and 2) also 
affected measurements. 

Evidence for Learning . Mean performance on the Part I criterion dealing 
with pt^oblem solving averaged about 87% for students who completed all instruc- 
tion (including J^Ixperiment 2) and 81% for all who took the test. On the Part II 
problem-solving cr^iterion, the average was about 8::'?^^ for finishers and 72% for all 
students. Thus, the program taught about as effectively as it had in Experiment lA. 

Since the mean performance scores suggested that the finisiiers were more 
homogeneous, the analyses of all experimental findings reported Ik-Iow are con- 
fined to them^ in order to provide uniformity in the data l^ase tfu (MiL^i\o-]t iUv 
study. Mean criterion and retention test scores are shown in 'I'ahlr ! C 

Prompting and Confirmation Effects , The Prompting Group sco: i i-nifi- 
cantly higher than the Confirmation Group (p<-001) during traiiuHL^ on lioth 
Part I (96.5% vs. 88<5%) and Part II (91% vs/S7%), The groups did rioi differ 
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Meon Criterion and Retention Test Scores 
of Naming/No-Naming Groupsi Experiment IB 
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Table 6 



Mean Criterion and Retention Test Scores of 
Prompting/Confrrmation Groups: Experiment IB 
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significantly on the two criterion tests; however, when scores were adjusted 
for learning error the Confirmation Group was superior (p<,005 for Criterion 
Test 1; p<.10 for Test II). Thir. adjustment vva*^ made by means of hin analysis 
of covariance; which provided an estimate of scores that would have been 
observed if performance during learning had been the same across conditions. 
The adjusted means for the Confirmation and Prompting Groups on both sets 
of criterion tests ^ire shown in Table 7, 



Toble 7 

Adjusted Mean iicores'' of Naming/No, Naming and 
Prompting/Confirmation Groups on Criterion Tests; Experiment IB 
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Verbalization Effects , Since the Rules Group was dropped from Experiments 
113 and 2, the re, laining groups were called the Naming (used mnemonics) and 
No-Naming Groups, depending upon whether they named rules and worked on 
practice problems, or simply worked problems during training. 

The two groups did not differ on the Part 1 criterion items dealing with 
problem solving, ITowever, on criterion items requiring students to name and 
write out rules, the Naming Group scored significantly higher, {The Naming 
Group had been given practice in naming rules but not in writing them out,) 
An analysis that took learning errors into account (analysis of covariance) 
showed that the superiority could be attributed in part to better performance 
during training (see Table 6), Retention tests reflected the same findings. 

Level of aspiration and intelligence data are presented in the final section 
of this chapter. 
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Kxperimonts 113 and 2 were conducted in succession, Experiment 2 dealing 
with tlic new and more difficult materials of Parts HI, IV, and V* The number 
of treatment coTnijinations wari quite high — 2£), Since attrition was also quite 
high — about half the students dropped out before the end of Part HI— many of 
the gt'oups in Experiment 2 were rather sparsely populated. Although this situa^ 
tion dampened the hopes of detecting statistically significant effects, those 
results which did prove reliable can be considered all the more powerful* 

[Evidence for Learning . Overall mean performance on the criterion test 
for Part HI \.vas about 7 0% eorrect; the median Was about 78%. Thus, there was 
evidence of learning at a point three to five weeks after training had begun under 
l:^xpcriraent IB. Overall proficiency dropped to a median of 64% on Part IV and 
i'35% on Part V. This drop was not unexpected since the purpose of the experi- 
ment wari to obtain comparative data on teaching combinations; many of the less 
effective coml>inations were quite ineffectual at the more complex levels. One 
condition — 7.ero confirmation— however, was effective and yielded medians of 
81%, 73%, and 48% on Parts IIL IV, and V. 

A three-way factorial analysis was performed on Partlll criterion data 
by randomly eliminating subjects to equate cell sizes* No interactions were 
revealed* Consequently, all subsequent comparisons considered the variables 
as independent and utilized all the data. The first set of analyses below treats 
Parts III, IV. and V separately* A second series was performed treating the 
three parts as replications* 

Prompting and Confirmation Effects * Prompting and confirmation are 
similar in that they both present information pertaining to the correct answer* 
The amount of information presented will be called the degree of stimulus 
support, whether it comes from prompting or from confirmation. It had been 
hoped that errors during training could be manipulated by varying the percent- 
age of stimulus support. An analysis of data from Part III indicated that this 
was accomplished. As shown in Figure 4, errors ranged from 10% for zero 
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Relations Between Learning Error and Criterion Test Error Under Pronriptrng 
and Confirnriation Conditrons: Part III 
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stimulus support to 2% for complete 
support {total pronipting). Perform- 
ance during training was directly 
rclnted to degree ofstimulu^i support; 
the greiiterthe suppot^tthe fewerthe 
errors. Howeverj test performance 
was invers el y related to stimulus 
support (p<.01) when scores were 
adjusted for learning errors. Test 
scores for the adjusted data are 
shown in Figure 5. 

The data showed marked 
variability in Part IV. The overall 
median dropped to 64% correct; the 
mean fell to 55%. In Part V the test 
scores were even lower, 38% correct 
as the mean and 35% as the median. 
_1 Therefore, modified nonparametric 
lOOP runs tests were employed using 
deciles instead oi" individual scores. 
Results supported the findings in 
Part III* Where confirmation and 
prompting reflected comparable 
stimulus support— all of the Zero Prompting Groups vs. all of the Zero Confirmation 
Groups, and all of the 100% Prompting Groups vs* all of the 100% Confirmation 
Groups— the confirmation treatment proved more effective for criterion perform- 
ance. A decile analysis of the criterion data relevant to these comparisons 
from Parts m and IV yielded significant differences (p < .01)* in Part V the 
Zero Confirmation-Prompting comparison gave the sam"e result, but the 100% 
groups did not differ. 

On the final retention test, given four weeks after completion of all 
instruction, the direction of the effects was the same. The lower the stimulus 
support during training the better the scores* However, significant differences 
were obtained only among the confirmation levels (p < .001}* A decile comparison 

among the prompting conditions 

Table 8 
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revealed that both the Zero and 33% 
Prompting Groups scored signifi- 
cantly higher than the 100% Prompting 
Croup (both with p<. 01). The other 
intermediate prompting groups did 
not differ from the Zero Prompting 
Group, but did score significantly 
higher than the 100% group. In short, 
the long-term retention test further 
indicated that the greater the error 
during learning the better the perform- 
ance on criterion problem solving. 
The retention test means and medians 
are presented in Table 8* These data 
are not adjusted for learning errors. 

"Variety -of-Practice Effects . 
There was little difference in learning 
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between the VLu-icty and No- 
Variety Groups during training. 
However, students who had 
practieed under the Variety 
condition scored significantly 
higher on the criterion test for 
Part III (p<.01; Figure G). 
Analyses of criterion test data 
from Parts IV and V did not 
reveal any significant differ- 
ences. However, the Variety 
students; scored higher on the 
Part IV test on a deciles com- 
parison (p < ,01), On the Part V 
criterion test and on the final 
retention test the effect disap- 
peared completely, although 
even in retention the direction 
of the difference still held {Vari- 
ety mean, 44%; No-Variety, 38%) 
Since the Variety stu- 
dents practiced with three 
different verbal contexts 
during training (vacuum tubes. 
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Relation Between Leorning Error and Criterion 
Test Error Under Variety-of -Practice 
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salaries, and inventories of uniforms) and there were four different contexts 
used In the five criterion problems, it could be argued that their superiority 
derived from a greater similarity i^etween training and testing than was the 
case with the No- Variety Group, which practiced on vacuum tube problems only. 

This possibility was tested by dichotomizing the criterion problems 
into those that were stated in a novel context for both groups (4 problems) and 
those Ltsing a setting new only to the No-Variety Group (2 problems). The 
Variety Group was superior on both types of problems* Thus, it can be inferred 
that variety of context resulted in better learning of the general mediating skills 
necessary to write computer programs, and not simply better learning of the 
overt associations practiced during training. ^ 

Verbalization Effects . Due to clerical error in assembling grouping of 
course materials, and to attenuation of some groups because of high dropout, 
no meaningful data were generated from the treatments added to get supplement 
tary information on the effects of verbalization, 

RELATIONSHIPS BETWEEN EXPERIMENTS IB AND 2 

Since Experiments IB and 2 were conducted in succession, using the same 
students and teaching a continuity of subject matter, it became important to 
determine the relationships between the two studies. Results presented below 
deal with the question of how various learning conditions in Experiment IB 
combined with learning conditions in Experiment 2 to affect the criterion test 
performance in Parts III, IV, and V. The two variables of Experiment IB and 
the three of Experiment 2 yielded six combinations of interest: 



Experiment IB 

Prompting/ confirmation with 
Naming/no-naming with 



Experiment 2 

Level of confirmation 
Level of confirmation 



25 



Prompting/conrirmalion with 

Naming/ no- naming with 

Prompting/conrirmation with 

Naming/no- naming with 



Level of prompting 
Level of prompting 
Variety/ no-variety 
Variety /no -variety 



Since there were no significant interactions among the variables of 
Experiment 2, they were treated as independent for purposes of analysis. Data 
from Experiments IB and 2 were cast into three-way analysis of variance tables 
to note possible interactions. Parts III, IV, and V were treated as replications. 
Graphs of the main effects of confirmation levels and prompting levels are 
shown in Figures 7 and 8 respectively (see also Tables 9 and 11, Analyses 
of Variance). 
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Naming/No-Naming (Exp. IB) With Confirmation Levels (Exp. 2) . The 
analysis of the naming and no-naming conditions of Experiment IB combined 
with the three confirmation levels of Experiment 2 is summarized in Table 9. 
Significant interactions were indicated. The means for the various treatment 
combinations are shown in Table 10 grouped according to a Duncan Range anal- 
ysis, which enables a comparison of individual treatment means to be made. 



Toble 9 

Analysis o( Voriorice of the Effects of Cor)firmotion [£xperfmer)t 2| 
by Naming/No-Naming (Experiment IB) on Criterion Tests 
for Parts III, !V, ond V 
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Duncan Range Groupings of the Effects of Confirmation (Experiment 2) * 
Naming/No.Naming (Experiment IB) Combinations on Criterion Tests 

for Parts III, IV, and V" 
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Table 10 reaffirms the stimulus support effect noted previously, that 
the lower the stimulus support during training (here, the lower the confirmation), 
the better the criterion scores. This effect appeared to be heightened when 
combined with prior use of mnemonics (naming the rules); practice in naming 
rules in Experiment IB plus low confirmation in Experiment 2 seemed to promote 
the best criterion performance. The effect is clearest in Partlll where learning 
began to deteriorate with increased difficulty in the instructional materials* Here 
the No- Naming/Zero Confirmation Group came in first, the Naming/Zero Con- 
firmation Group second, and so on, with the No-Naming/l 00% Confirmation 
combination representing the worst learning conditions* 

Prompting/Confirmation (Exp* IB) With Confirmation Levels (Exp* 2) . The 
analysis of the prompting/ confirmation conditions of Experiment IB in combina- 
tion with the various confirmation levels of Experiment 2 revealed no significant 
interactions* Examination of the individual groups did, however, suggest a type 
of interaction effect related to similarity of stimulus support. It appeared that 
the more similar the stimulus support conditions the better the performance 
(see Figure 8). Specifically, zero confirmation (the Prompting Group of Experi- 
ment IB) followed by zero confirmation (in Experiment 2) resulted in better 
scores on criterion tests for Parts III, IV, and V than total confirmation followed 
by zero confirmation* This relationship also held for the zero-to-total vs* 
total-to-total confirmation sequences except for Part IV, where there was 
a reversal* 

Naming/No- Naming (Exp. IB) With Prompting Levels (Ex p* 2)* Analyses 
of the data (Table 11) reaffirmed the importance of low stimulus support (here, 
low prompting levels) and early practice with mnemonics (naming the rules) to 
criterion p^^rformance on Parts III, IV, and V* Mean scores resulting from the 
various sequences of treatments. Experiment IB to Experiment 2, are shown in 
Table 12, grouped according to the results of a Duncan Range analysis. In gen- 
eral, naming followed by low prompting promoted the best performance; no- naming 
followed by high prompting was the worst combination* 
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Analysis of Variance of the Effects of Prompting (Experiment 2) 
by Namlng/No- Naming (Experiment IB) 
on Criterion Tests for Parts 111, IV, and V 
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Table 12 

Duncan Range Grotipings of the Effects of Prompting [Experiment 2] - 
Naming /No. Naming (Experiment IB} Combinotfons on Criterion Tests" 

for Parts III, IV, and V 
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Prompting/Confirmation (Exp. IB) With Prompting T^evels {Exp, 2) . The 
Lnieracixon of early prompting/confirmation treatments (Experiment IB) with 
later promj)ting levels {Experiment 2) way highly pronounced (Table 13)* The 
dominant factor appeared to be similarity of stimulus support, as It was in the 
previouir discussion of prompting/confirmation followed by confirmation. Thus, 
as seen in Figure 9, students given total prompting in both studies performed 
better on Parts III, IV, and V than students; given zero prompting followed t>y 
total prompting* Similarly, zero prompting in both experiments promoted Ijettcr * 
performance on all three criterion tests than total prompting followed by 
zero prompting* 

er|o 



Toblo 13 



ERIC 



Analysis of Variance of the Effects of Prompting (Experiment 2) 
by Prompting/Confirmation (Experiment 16) 
on Criterion Tests for Parts 111, IV, and V 
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Effects of Similarity of Stimulus Support From Experiment IB (0 or 1001 to 
Experiment 2 (0 or lOO) on Criterion TesI Performance: Experiment 2 
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Naming/No-Naming (E>:p, IB) With Variety Level(E>:p. 2) . Analyses showed 
that criterion performance in Parts III, IV, and V was clearly affected by the 
various combinations of early naming/no-naming with later variety/no-variety 
(Tabic 14)* Although the data were not entirely consistent, it appeared that 
performance improved as the combination approached the mix of early naming 
with later variety* r^irtlicr, the importance of naming seemed to increase us 
materials became more complex (and, concomitantly, as performance dropped 
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Analysis of Variance of the Effects of Variety/No. Variety 
(Experiment 2) by Naming/No-Naming (Experiment IB) 
on Criterion Tests for Parts lllr IV, and V 
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off), while the importance of variety s eenied to decrease. Means oi' the various 
combinations are shown in Table 15, arranged according to the results of a 
Duncan Range analysis. 

Prompting/ Confirmation .Exp, IB) With Variety Level (Exp, 2) , No signifi- 
cant interactions were found* 
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Duncan Range Groupings of the Effects of Variety/No-Voriety (Experiment 2) - 
Noming/No^Noming {Experiment IB) Combinations on Criterion Tests" 

for Parts IIL IV, and V 
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FACTORS R1!:LATED TO DROPOUT 

As noted earlier, about 57% of the students did not complete instruc- 
tion, A series of analyses were conducted to determine the relationships 
between dropout and stimulus support, level of aspiration (LOA), intelligence, 
and performance. 

Stimulus Support , The more stimulus support in training— that is, the greater 
the degree ofprompting or confirmation— the greater was the likelihood a student 
would finish the course (x^ trend test of proportions, p<,05)* Figure 10, based 
on Part III data, shows that the prompting and confirmation factors operated 
alike in this regard. 

Level of Aspiration , At the end of Parts I and II, students were asked to esti- 
mate how well they had performed ontheprevious part and how well they expected 
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to perlbrni on tlu; instruction to follow, 
Tlio cori*ol;\tjons between performance 
estimates on completed intitruction 
and actual performance wet^e quite 
higli (.CTj). ITowevet^ studentt? ^'ho 
later di'oppod out expected to per- 
form much higher on subsequent 
c rite rnontessts than they did, relative 
to .students who finished (p < .01), That 
is, the discrepancy between level of 
aspi ration and performance was much 
higher for dropouti5 than for finishers. 
In addition, the expectations of the 
dropouts wore significantly lower 
than the expectations of the finitihers 
after Part I, and the performance of 
the dropouts was significantly lower. 

In short, relative to the 
finishers, the dropouts (a) expected 
to perform at a lower level, (b) greatly 
overestimated their capability, and 
(c) did perform significantly worse. 

Intelligence. Dropouts were characterized by significantly lower intelligence 
(verbal ACB scores) than finishers. 

CORRELATIONS BETWEEN EXPERIMENTS IB AND 2 

Experiment IB , The overall correlation between intelligence and criterion 
performance was high (Part I, r=.C9; Part 11^ r=.70). The relationships were 
even higher for finishers {Part I, r = .73; Part II, .73). 

The overall correlations between learning errors and scores on the 
related criterion tests were not so marked, but increased from Part I to Part II 
(.4Cto.C8)* Forthose subjects on whom intelligence measures were not obtained, 
the relationship between learning and testing was even higher (.48, N= 141; .32, 
N= 140). Since learning errors fell within a narrow range (2%-8%), these cor- 
relations may be spuriously low* 

On dividing students by intelligence into low, medium, and high, it was 
found that the learning-testing relationship was significant for the average and 
bright students who dropped out, but not for the average and bright students who 
finished. For low aptitude students the correlations foetweenlearning errors and 
criterion scores were about the same, positive but low (Table 10). 

The correlation between the criterion tests for Parts I and II was high, 
about .7. Thus, performance on the first criterion test was3 a good predictor of 
success on the second* 

Experiment 2 * The partial correlations, for finishers only, between LOA 
(expectations), intelligence, and criterion performance are given in Table 17. 
The relationship between LOA and subsequent criterion performance, with intel- 
ligence scores partialled out^ increased (except between Parts IV and V) as 
students progressed through the course. On the other hand, overall relationships 
between intelligence and criterion scores decreased through the instruction. The 
relationship between LOA (e.^pectations) and intelligence decreased even more 
markedly, shrinking to almost zero (+ *05) on the last criterion te.st. 
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Correlations by Intelligence Level Between Learning and 
Criterion Scores of Finishers and Monfinishers; 
Experinnent IB 
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CorrelaHons Between Expected Success, Intelligence, and 
Criterion Performance of Finishers, as o Function 
of Course Complexity: Experiment 2 
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In short, as the finishers progressed through the course^ the relation 
between instructional-based factors — LOA and performance— increased; however, 
the relationships bet^veen these factors and Intelligence, a measure taken before 
instruction began, decreased. 
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DISCUSSION 



The organization of the ensuing discussion will be from specific to general, 
and from experimental hypotheses tested in Experiments lA and IB to those 
tested in Experiment 2. Following material pertinent to the immediate findings, 
the implications of this type of study for future research and development in Pi 
will be discui;i;ed. Finally, the potential utility of the secondary product of the 
MEXrrOD II research, the programed course in fundamentals of computer pro- 
gramming, will be indicated. 

COMPARISON OF EXPERIMENTS lA AND IB 

Evidence for Learning 

The means for percent correct on the criterion tests on both Parts I and u 
were almost identical across studies: 87% on Part I and approximately 83% on 
Part IL The retention test scores for Experiment lA are somewhat higher 
than for IB; however, the test for Experiment lA was given on the dayfoUowing 
criterion testing, ^v^hereas in Experiment IB a week separated criterion and 
retention testing* 

The intelligence indicator was significantly and positively correlated 
with criterion scores in both studies and no interactions appeared between 
intelligence and treatment conditions on the principal measure, writing com- 
puter programs* 

Treatment Eftects 
Learning Time 

Unfortunately, administrative problems prevented obtaining adequate 
time measures in Experiment IB. In Experiment lA, length of learning time 
was negatively related to score on the criterion. Doubtless, this was due to 
the time spent by the Rules Group in writing out complex programming rules, 
a requirement which appeared to dull their terminsd performance* The learn- 
ing time/criterion score relationship may not generalize to other PI environments. 

Criterion Scores 

(1) Prompting/Confirmation Results . The tendency in Experiment lA 
for the prompting/confirmation effect during learning to be reversed on testing 
was found even more strongly in the oovariance analysis under Experiment IB. 
Learning errors in the two studies were highly comparable: 3% and 3.5% for 
prompting, 10% and 11.5% for confirmation. 

(2) Verbalization Results . With respect to the requirement of writing 
the names of the rules during training (mnemonics) as opposed to just writing 
computer programs, the suggestion in Experiment lA of superiority of the 
Naming Group with the more complex materials in Part II was heightened in 



Experiment IH, wliicli involved Tive times as many students. Moreover, the 
superiority of the Naming Group (mnemonics) in the lat^ger study was in the 
percent cot^rect |30r .se on the criterion in Part H and did not involve time- 
weighting the scores* That irf, use of mnemonics resulted in an absolute and 
not just relative superiority, which carried over a two-week period, adding to 
the practical value of this particular finding. 

Summary 

Summing up the effects of tlie treatment conditions aeross both experiments, 
the use of mnemonics in learning the concepts in the computer pr^ogramming 
course aided students in tests of computer programming. On the other hand, 
practice in writing the entire rule in the context given by the booklet hindered 
ability to use the rules effectively. 

The confirmation and prompting comparisons are not so clear. Taken 
togetlier, tiie two studies reveal that while confirmation led to more errors 
during learning, it did not lead to differences on criterion testing. If, however, 
one takes into account the covarlate of errors during learning, then confirma- 
tion was vSuperior to prompting (indicating that the Confirmation Group benefited 
from the errors made during training). Nevertheless, one cannot say that in 
the context of these two experiments error rateduringlearningwas an Important 
factor in criterion test performance, 

EXPERIMENT 2 

Stimulus Support 

It seems clear within the context of the present research that a low degree 
of stimulus support (in the form of problem information), with attendant relative 
high degree of error during training, is desirable for achieving a high degree of 
proficiency on a criterion test requiring the synthesis of the materials presented 
during training,* Variations in degree of prompting or confirmation were 
inversely related to criterion performance, 

,^t the outset it was thought likely that a certain amount of error during 
training would result in the best criterion performance, A legitimate question 
is: Why did not this optimization relationship turn up? 

,^lthough the reasoning is post hoc , a good starting point is to note that the 
key to good instruction is transmission of information. Since error rate in the 
worst experimental variation of the course was still low, approximately 13%, it 
is proposed that all groups were either at or beyond the hypothesized optimal 
point relating information transmission, difficulty level, and test performance. 
To explore this assumption in future research, it would be worthwhile to degrade 
the information transmitted by decreasing the chunks cf conceptual content 
presented at each step of training and relate these variations to end of course 
performance (see Seidel, 21, on the problem of units of meaning). 

These findings support the proposition that complex human learning such 
as problem solving is of a different nature than the rote learning of isolated 
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paired associate!? or serialized items. The latter involves tralnini^ to :\n inter- 
nal achievement criterion, as opposed to the problem-solving ci^ittMn^Mi v.luch 
is external and of such a nature o.^ to require synthesis or the concepti;; niKl the 
development of a strategy in a transfer test. Perhaps a good point oT departut^e 
for the research reported hor^e is Kendler's I't^^tont state, jient (12, p, 231): 

If researchers in the fieldof programed learning belie\''e that tht^ major 
response being learned is the one that the student writes in Hu^ Cramej 
they are badly mistaken. What is being learned ar^e concepl:^ ■ i whixt 
are modiTied are large segments of the students' veriial repui'iui ix-. 
It sihould be noted tliat, when prompting or confirmation wore given in the present 
set of experiments, they were applied not to simple stimulus-response associa- 
tions (as in rote learning) but to strategies or portions thereof* 

In oi^der to be appropriately applied in the current context, Kendler's state- 
ment should be amended to include that not only are concepts being learned but 
strategics enabling the student to use these concepts c^re being learno<l as well* 
Thus, he reaches a higher level of learning called problem solving. This ii^ in 
agreement with Gagnfe's recent distinction (13) between concept learning and 
problem solving,^ 

Variety 

The overall approach of the present research can be meaningfully compared 
to a similar study by Gagne and Bassler (^3), who used a programed Instruc- 
tional environment to study teaching and retention of elementary non-metric 
geometry with sixth graders. Despite certain procedural differences, concep- 
tually the approaches are Sufficiently similar so that a detailed comparison of 
results should indicate some possible generalizations as well as being quite 
instructive for future research. 

To begin with Gagne and Bassler found a moderate correlation — of the order 
of +.62 — between achievement scores immediately following learning and those 
of retention. In the present study^ a comparable relationship (of the order of 
+.64) was found. In the study by Gagne anJ Bassler, they were able to compare 
the subordinate learning sets immediately following learning and in retention 
and found a correlation of -^.46, Correlations in the present study for the early 
or subordinate criterion tasks were on the order of 1,00. Whereas Gagne and 
Bassler found that the correlations between the retention and theoriginal achieve- 
ment scores for the subordinate learning sets were lower than for the higher 
order learning sets, the data in the present study indicated no such difference. 

These findings are reflected in another way by looking at retention ratios 
(retention score divided by original score) for the various problems. They are 
all just about 1*0 or fairly close to it. Differences which did appear were con- 
fined to the effects of different degrees of prompting in Parts III, IV, and V; the 
mean ratios for the latter are given in Table 18. The differences occurring on 
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Problem 4 in that table and on the overall retention problems among the various 
prompting conditions are significant* These effects generally reflectthe original 
effects of the stimulus support variables, showing the least stimulus support 
(zero prompting) yielding the maximum percent correct and the 100% stimulus 
support showing the least percent correct {F = 2,9C, dr^4/l74, p < , 05 for 
Problem 4; c)f^4/l74j p<»05 for overall retention comparison), 

Gagn§ and Bassler found that » » the present evidence suggests that their 
[the subordinate learning sets] importance diminishes once the final task has 
been achieved" (23^ p» 127). However^ in the present study the retention results 
indicated this was not the case, that the subordinate concepts were retained and 
had the same effect as they did on the initial achievement* 

Contrary to the Findings of GagnS and Bassler, while the results of the 
present study showed the effect of Variety on immediate achievement^ the effects 
diminished to nonsignificance on the retention test four weeks later* Their data 
yielded just the reverse effect^ but it should be noted that in their study achieve* 
ment was measured after 11 successive days of trainings whereas in the current 
study final achievement was measured after appro:vimately 10 weeks of study^ 
one Session per week. Retention in their investigation was measured nine weeks 
after the immediate achievement test^ whereas in the present instance retention 
measures were taken four weeks following the last achievement test* Thus, it is 
possible that differences in the results may be due in part to the differences in 
the time frames for instruction and retention testing* 

Mnemonics Training 

Another aspect to the results of the current research relates to the value 
of written verbalization of rules which an individual is required to use in develop- 
ing his strategies for problem solution. The data of both experiments taken 
together clearly indicate that "what is in a name" is important* If an individual 
IS given mnemoniC5> practice early in training and required to learn these 
mnemonics as well as to use them in forming his strategy (writing computerpro- 
grams)j he is able to formulate the rules for later use with greater facility than 
if he is required to writethe rules inthe verbal context given by the intstructor* 

This makes good sense if one thinks about how we go about encoding our 
environment from day to day* If an individual is asked to recall zin event, it is 
much easier for himto provide a sentence or two inhis own words than to report 
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in great detail or verbatim (if the event is a poem or story). In like manner, 
wlien requiring a student to learn the conceptual rules for the use of computer 
cojnmands, if he is provided with a mnemonic key with which to tag the rule as 
he formulates it himself, he will be able to use his own verbal repertoire to 
good advantage. On the otlicr hand, if the student is required to use the instruc- 
tor's words-^vvhich may not match his own-^in formulating the rule, he may 
indeed end up just memorizing the words without developing any understanding 
in the use of these rules. At least, that seems to be the inference to be made 
from our present study. 

It is this last feature which seems to take exception to one of Gagn&'s 
positions (13^ pp. 307-308) in which he emphasized the value of recallability 
of "subordinate capabilities." The present study clearly supports a distinction 
between recallability and availability of these subordinate abilities, the latter 
being the more important of the two. The Rules Group in Experiment 1 A recalled 
the rules much better than the other groups, but they did so at the expense of 
being able to use these lower-order skills. On the other hand, the Naming Group 
was not only able to recall the rules well, but was able to have them available 
for use in working the computer programs. 

This distinction between recallability and availability receives further 
support from Torrey's recent work (24) on the teaching of Russian grammar. 
She found that students given contextual drill during learning were better in 
dealing with new material on a generalization test than were the controls^ The 
latter students learned all the training material by a vocabulary method. It is 
important to note that the control subjects were periodically tested to show their 
ability to generate correct sentences according to the Russian grammar rules. 
Thus, during training they were able to generate the rule, but on the test they 
were not able to apply it. 

The value of mnemonics as used in the present study (Naming Group) 
warrants further discussion^ As already noted, naming practice was helpful 
in solving computer problems related to the course material directly involved 
(Part 11 criterion). Of perhaps greater significance were the more lasting 
advantages which appeared in later and more complex portions of the course. 
When overall proficiency dropped to alow level (Part V and final retention 
test), a significant interaction appeared between early naming practice (Parts 1 
and II) and the later treatments (Parts III, IV, and V). This took the form of 
good early training in Experiment IB (Naming) compensating for poorer train- 
ing conditions in Experiment 2 (see Tables 10, 12, and 15). 

For example, the No- Variety students who had early training with mnemonics 
were able to maintain as high a level of criterion proficiency in Parts IV and V 
as the Variety Group. In other words, the rate of loss of proficiency was not 
as great for the No-Variety Naming Group as it was for the Variety No-Naming 
Group. The compensating effect was also superimposed on the stimulus support 
treatment effects as seen in Tables 10 and 12. 

A reasonable question which follows from these results is: What is the 
relationship between stage of training in which mnemonics are introduced and 
enhancement of criterion performance? An attempt was made in Experiment 2 
to obtain ^some relevant data by continuing the use of mnemonics through the 
more complex portions of the course for three additional groups of subjects* 
Unfortunately, because of clerical error in the preparation of the booklets, and 
group attenuation because of drop-out, meaningful data were not generated. The 
question of timing and relative value of mnemonics, of course, remains an open 
and important one for future research* 
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Further discusJsion of the transfer efFects of general vs, task-specific 
factors is worthwlule because the subject haa special relevance to the design of 
instructional programs, Aa seen in Table 17, the initial intelligence measure 
is of decreasing valuo i\s a predictor of succeeding criterion porformance* 
Conversely, the successive criterion test data and level-of-aspiration measuros 
increased in predictive value. These findings suggest {as does Bunderson, 25) 
that a general intelligence (or reasoning) factor may bo of greatest importance 
only for early stages of instruction. Thus, in constructing instructional models 
for optimizing a student's path through a course, the Implication is that woight- 
ing for general and task-specific factors should be differentially assigned and 
continually checked (to decrease the formec-arrd increase the latter) at appro- 
priate choice-points during instruction, 

PRACTICAL POTENTIAL OF THE PROGRAMED COURSE 
Relationship of Content to the Parent Army Course 

In order to evaluate most appropriately the practical potential of the pro- 
gramed course, it would be desirable to be able to state the number and percentage 
of hours covered relative to that required by the total ADPS prog ram mirg course 
presented at the Signal SchooL However, because the material selected was 
chosen to represent the basic fundamentals of computer programming and 
because the authors' frame of reference for organizing did not necessarily 
coincide with that used by the instructors in the ongoing course, it would be" 
extremely difficult to try to represent the finished programed instructional 
product in this manner. As an alternative, it may be helpful to examine the 
relationship between the conceptual material presented in the programed 
course and in the total ADPS course at the Signal SchooL 

Concepts covered in the programed course deal with internal data handling 
of the central processing unit(CPU), Also included is materlalondesk-checking 
and flow-charting. No input or output instructional materials were included. 
The number of commands included (11) represent roughly 27% of those in the 
parent course dealing with internal data manipulations. Viewed in terms of the 
proportion of such conceptual categories represented, the programed course 
includes approximately 50% of those given in the ADPS course. Finally, if the 
basic logic of internal data handling is considered (i,e., within the CPU), the 
programed course includes virtually the entire content* Thus encompassed are 
the simple movement of data from place to place in storage {although not without 
moving to an intermediate centi'al location), arithmetic operations, comparing 
and sorting (general use of the loop concept)^ and some sophisticated program- 
ming techniques (e,g., use of index registers). All except the last categorywere 
involved in the experiments conducted with the programed course. 

Administration, Motivation, and Population Factors 

Certainly the large percentage of dropouts among the volunteer subjects in 
Experiments IB and 2 (57%) prevents a firm conclusion about the practical 
potential of the programed course* That motivational factors as well as course 
difficulty might have been the cause for the number ofdropoutswas supported by 
the level -of-aspirationdata (the differences in realistic expectationbetween those 
who dropped out and those who remained in the course) and the importance of 
the stimulus support (prompting or confirmation) factors as determinants of 
tendency to drop out. 
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A fin^the^r complication to determining the course's practical potential was 
tlic Tact th.il the administrative conditions under which the course was given 
wei^c not thorio which obtain in the real world, whether it be in ^n Army school 
or ^ civilian school setting. Specifically, the students were volunteers and 
T^eccivcd no incentive in the larger study (Expcr^imcnt 11^ and 2) e:fcept a 
certificate at the end of the 14"Week course indicating proficiency in the HumRRO 
compLitLM* progi^amming fundamentals course* In Experiment lA, on the other 
hand, the students met for three successive afternoons for a maximum of four 
hours per afternoon and were paid for their parti cipation* Interestingly enough, 
these students performed better than did the other high school students in 
Experiments IB and 2 in the comparable portion of the course* 

Finally, the students in the large-scale study may not have been as intel- 
ligent as the target population of Army students (the mean ACB CL score of 
the high school students was 06, whereas the Army requirement for"acceptance 
in the ADPS Course is 110). 

After the large-scale experimentation, it was decided to investigate further 
tlie possible contribution of motivation, administrative procedures^ and course 
difficulty to outcome by administering the course to a sample of enlisted person 
nel at Fort Gordon, Georgia (N = 25)» Such men presumably would be more 
representative of the target population for whom the course was originally 
constructed* The men chosen for the experiment had just finished a teletype 
operator's course (MOS 723)' and were waiting to be shipped to their duty 
assignments* They were given six hours of instruction per day for five suc- 
cessive days* Their incentives were ball-point pens and cigarettes, allocated 
according to degree of proficiency. As in the large study, roughly half the stu- 
dents failed to complete the course, although again the general attitude on the 
part of the students toward the course materials was favorable* They felt that 
they would like this kind of course presented with the aid of an instructor. As 
in the larger study, the ACB CL scores of the dropouts were virtually identical 
to the scores of those who finished the course* 

Instructional Effectiveness 

When attempting to evaluate the instructional effectiveness of the course^ 
one must keep clearly in view the fact that the primary purpose of the overall 
study was to investigate a wide range of training techniques* Thus, it would be 
expected that only a few of the combinations would yield adequate proficiency, 
particularly as the training materials and criterion requirements became more 
complex* Thus, although the Duncan Range analysis (Table 10) appears to show 
very poor performance on the most complex portion of the course (Part V). the 
combination of lowest degree of stimulus support (especially 0% confirmation) 
still yielded a criterion mean close to 50% correct. The medians of the zero 
confirmation conditions (N = 79) showed 81% correct on Part HI; 73% on Part 
IV; and 48% on Part V. A further breakdown into variety or no-variet^j and 
degree of prompting as well as interactions with the treatments of Experiment 
IB is not valuable since the number of subjects within each of these smaller 
groups would be much too small to yield meaningful data* Nevertheless, one 
can make a reasonable inference that at least some of these combinations would 
lower the median score for the entire 0% Confirmation Group averaged across 
all of these conditions* 

^Undcr the ciirrertt MO>S strut Hire* the teletype operators t ille is (^hmniunicnlioni? ('enter Specialist* and 
his MOS is 72H. 



The tentative inference, then, is that the instruction was effective* but 
because of the overriding experimental goals of the study, coupled with the 
other factors noted herein, no exact statement is possible. Although no hard 
data are available on attitudes, it has been the impression of the course adminis*- 
trators and proctors thai general rcat^tions towiird the programed Instructional 
nature of the materials were favorable. 

While it seemed apparent that revision was required to simplify the course 
material, it was not completely clear that motivational factors could be ruled 
out as contributory to the resulting performance of the experimental students. 
One final point with regard to the contribution of administrative procedures to 
performance should be made. Because experimental control over student read- 
ing of the materials was of primary interest, all students were required to read 
one page at a time and not to turn back to previous sections whenever they were 
in difficulty. Obviously in a practical setting, the opportunity to review material 
already covered would be desirable. IJT in the experimental setting it would have 
been possible simply to measure the number of times a student returned to a 
previous page or, for that matter, turned ahead to other pages, it would have 
been perfectly acceptable to allow it. Because of the need for experimental 
control, however, and the impossibility of measuring the review the student 
might do, it could not be permitted. 

FINAL REVISION OF THE COURSE 

As a result of information obtained from the experimental situations, the 
materials were simplified and certain format features were added. The final 
version also incorporated implications from the experimental findings and the 
course was then completed by including a section on the use of index registers 
as the most complex portion of the instruction. 

Description 

In the revised course, the entire context remains oriented toward the writ- 
ing of computer programs. That is, a concept is introduced and then a problem 
incorporating this concept is presented for the student to program. These 
problems were taken from actualcomputerprogrammingproblems encountered 
on the job. It is this problem-oriented approach in the course that makes it 
rather different from most of the other programed booklets available. 

Before each concept is presented, the student is given a preview of things 
to expect. Then, as he is introduced to the concept, he is given short completion 
questions with the answers in light type on pages opposite the questions (see 
Appendix C for sample pages). The problems to be programmed appear in a 
box-like format and the student is expected to answer the problem^ whenever 
possible, without looldng up the answer first. At the end of each section, the 
student is given a review and then led on to the next set of concepts. 

As was noted previously^ because a secondary objective of the research 
was to produce a programed course^available for use by the Army, the reper- 
toire of commands and the symbolic language in which the computer programs 
are written have been chosen from material previously used by the Army to 
teach students how to program the MOBIDIC computer. The language— the 
FIELDATA Symbolic Language— not only represents a setof symbols interpretable 
by the MOBIDIC* but is sufficiently general that it largely applies to almost any 
other computer and to the English language (the addresses). 

The Introduction to the course provides an overview of the concepts of 
computerprogramming and covers concepts of command and sample mnemonics. 
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address, the accumulator, memory locations, input and output, and computer 
functioning. There are periodic review materials and questions for the student 
and, finally, a short sample program. 

The course proper* is separated into lour levels: 

(1) Basic Opei"\tions (five sections), includes an introductionto storage 
locations, the concept of a central work area, the movement of 
numbers from place to place^ and simple arithmetic operations. 

(2) Basic Looping concepts are covered next (five sections}^ including 
general transfer commands^ counting the loops, leaving the loops, 
and program preparation* 

(3) Data Processing (four sections) deals with address modification 
and address arithmetic (effective addressing), sorting and counting 
data into various categories, and multiple address modification. 

(4) Advanced Techniques for transferring data are given (five sections), 
including inde^cing. 

In addition to writing many computer programs following the completion of each 
level, the student practices on problems within each section of the course. 

Evaluation 

The final revision of the course was administered successfully to seven 
college freshmen and si:c high school seniors* Of the college freshmen, four 
had scores of 90% or better on the final criterion test, two were in the high 
80s, and one scored 77%. Median completion time was 26 hours, with daily 
sessions self-paced and lasting from 3 to 5 hours* The high school students 
scored slightly lower, 78% to 88%, with median course duration being 31 hours* 
It should be noted that the two highest scores (88%) in the high school sample 
were obtained by students in the bottom third of their senior class standing. 
Again, the general attitude of the students toward the course was favorable 
and they felt they had learned something that would be useful to them* 

Apparently the revised version of the course was highly successful. It 
can be anticipated that the instruction would be successful in a military setting 
by virtue of the combination of experimentally effective training methods, 
simplified content, and administration procedures normally employed in mili- 
tary courses. However, no large-scale validation of the revised course to 
establish proficiency and failure rates in a military setting was undertaken, 
since this was not the principal purpose of the research.' 

Suggested Utilization 

The ultimate question is how the revised programed course could best be 
utilized. The nature of the course content is such that it would be quite useful 
for any officer or enlisted man who needs to have some familiarity with com- 
puter programming, whether for administrative or supervisory duties or some 
allied operation activity* 

The adequacy of the self -^contained aspects of the programed instructional 
course has not yet been evaluated on alarge scale. However, sincethe structure 
of the course Is geared to actually writing computer programs— to learning by 
doing— it should serve both as a useful introduction to a full-scale ADPS pro- 
gramming course and as supplementary instruction for students who may be 
having difficulties with those portions of the course covered in the programed 
instructional version. 

^Ooo polenlinlly useful finding related to prediclor vulidily wns the Uici thitL llie Verbul score on ihc 
ACH correlated much hr^^her with LTiierioti perforninnuc (r- k7) ihun llic (Ilcrftiil score (r ^*2.'>). The Army 
now uses a combined (A, store for course necept^nte. 
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SAMPLES OF COURSE CONTENT AND PROBLEMS 
FROM EXPERIMENT 2 



In these pages of sample course materials from Experiment 2, the material 
below and on the next two pages illustrates presentation of course content; the 
next three pages present three problems typical of the practice and test prob- 
lems given the students, and the final three pages show the answer sheets for the 
three problems* These materials are from the Variety/Confirmation condition. 
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Data Processing: MULTIPLE CATEGORIZATION 



Either TRZ or TRN can be used to sort addresses into two categories. But 
what if you want to sort into more than two piles? For example^ suppose that 
each address in a series has a 1^ a 2, or a 3 in it ^ and you want to know how 
many numbers there are of each type? The process is called multiple categoriza- 
tion. Here is a sample set of instinictions for the sorting portion of data 
processing. Notice that it is the same type of sorting used in the subtract- 
sort- and -count problems, but that it includes just one additional instruction-- 
the use of both TRZ and TRN. 



If negative, the number was originally a 1 

If zero after subtraction, the number was a 2 

If the number in the Accumulator is neither a -1 
nor a 0, it must have been a 3 before subtraction, 
and can be counted immediately following sorting. 

In other words, clear and add the number icopy it into the Accumulator) 
and subtract a constant which will make it possible to branch on TRZ or TRN 
(2 in this example) . 

1. If the number was originally a 1^ subtracting 2 results in a -1 in 
the Accumulator, which is a negative number. The TRN (TRans fer if Negative) 
command then transfers the program down to Symbolic Location TYPEl where the 
instructions which process the information coded by 1 's is processed. 

2. If the number was originally a 2, subtracting 2 results in a 0 in 
the Accumulator, a zero. The TRZ^ command then transfers the program to 
TYPE2. 

3. If the number was originally a 3^ the remainder in the Accumulator 
will be a +1, and instead of branching, the computer will perform the instruc- 
tions next in order* 





CLA 


NUMBER 




SUB 


TWO 


CD 


TRN 


TYPEl 


C2) 


TR^ 


TYPE2 


(3) 


CLA 


TRIO 




ADD 


ONE 




STR 


TRIO 



ERIC 
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t2S) [in coursel 



A supply depot maintains a two-word 
record on each vacuum tube in stock > stored 
in sequence relative to TUBE. The words 
are cost and location. The first word is 
coded with a 2 If the tube cost less than 
$5, with a 3 if it cost between $5 and $10, 
or with a 4 if it cost more than $10. 
Write a program to determine the number 
of tubes in each category. Store the 
answers in LOCOSTj AVCOST, and HICOST. 
The total number of tubes in stock is 
in TOTUBE, which can be saved for the 
loop counter by storing in temporarily 
in COUNT. 

Notice that data processing 
requires (1) subtraction of a constant, 
(2) sorting with both TR2 and TRN^ 
and (3) TRU instructions to the test 
for completion after both types of 
addresses counted at the end. 

Preliminary^Description 

Prepare COUNT 
Zero LOCOST, AVCOSTj and 
HICOST 

Subtract and sort : 
REPEAT CLA TUBE 
SUB THREE 
TRN LOW 
TR2 MEDIUM 

Count I's : 

add 1 to HICOST and store 

LAST test COUNT for completion 

modify TUBE 

(add 2 to REPEAT) 
TRU REPEAT 

Count -O^s : 
MEDIUM add 1 to AVCOST and store 
TRU LAST 

count -l^s : 
LOW add 1 to LOCOST and store 
TRU LAST 

STOP HLT 



Counter 
preparation 
0 items 



no 



Yes 



clean out 
garbage 



CLA address 
SUB constant 



count posi- 
tive numbers 



address mod, 
TRU loop 



Count zeroes 
TRU coropletion 



count nega- 
tive nuinbers 
TRU completion 




Yes 



Yes 
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28 [in course] 





TnTFltoF 
1 U J U DC 


TR2 


STOP 


STR 


COUNT 


CLA 


2R0 


STR 


LOCOST 


STR 


AVCOST 


STR 


HI COST 



REPEAT 


CLA 


TUBE 




SUB 


THREE 


2 


TRN 


LOW 




TR2 


MEDIUM 





CLA 


HICOST 


3 


ADD 


ONE 




STR 


HICOST 



CLA 


COUNT 


SUB 


ONE 


STR 


COUNT 


TR2 


STOP 


CLA 


REPEAT 


ADD 


TWO 


STR 


REPEAT 


TRU 


REPEAT 





LOW 


CLA 


LOCOST 


7 




ADD 


ONE 






STR 


LOCOST 


8 




TRU 


LAST 





NfEDIUM 


CLA 


AVCOST 


9 




ADD 


ONE 






STR 


AVCOST 


10 




TRU 


LAST 



STOP HLT 
It would be a good idea to 

ERIC 



First we prepare the loop counter to be used later 
in tlie test for completion by checking TOTUBE to 
guarantee at least one tube and storing it in COUNT, 

Since no other number needs to be saved for compu- 
tation, we next ^*clean out garbage^* from all 
locations to be used for storing answers by replacing 
any numbers that might be there with zeroes* 

Since the word indicating the tube's location con- 
tains a 2,.Z, or 4, we subtract 3 to make the 
numbers -1, 0, and +1, After the subtraction, a -1 
remainder indicates a LOCOST tube, which the TRN 
command will detect and transfer the program down 
to Symbolic Location LOW, where these tubes are 
counted. Similarly, a 0 indicates an AVCOST tube, 
which the TK2 command will pick up* 

If the number was neither a -1 nor a 0, the TRN and 
TRZ commands will not transfer the program* The 
remainder must have been a +1, indicating a HICOST 
tube* 

TOTUBE contains the total number of tubes to be proc- 
essed, but since we wanted to save that number, it 
was copied into COUNT, which we now use in the test 
Tor completion. 

Since each tube has a two-word record, we must add 
2 in address modification to get to the address 
containing information on cost for the next tube. 

Having reached the bottom of the loop, we transfer 
back up to begin processing of the next tube* 

The TRN command detected a -1, which means that the 
number was originally a 2 (before subtracting 3), 
and a 2 was the code number for a low cost tube. 

Since the program branched out of normal order, we 
must transfer back up to finish out the loop* 

A 3 in the word indicated an average cost tube. Sub- 
tracting 3 made it a 0, which the TRZ command 
detected. Having picked up an AVCOST tube, we count 
it here. 

All tubes must be accounted for in the test for 
completion, whether they are counted right after 
the subtract-and-sort or down here at the end. 



go over these two pages a few more times. 
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Problem 1 

The Base Electronics Warehouse wants a program that will count tlie 
number of tubes used in January of last year and the number used in February. 
The results are to be stored in JAN and FEB. 

The words for each tube are in memory starting at location VACUUM. The 
words are; part number, date received, shipment number, manufacturer, date 
tube was used, and place tube was used. TUBES contains the number of vacuum 
tubes in the warehouse. The month in which the tube was used is coded by 
number: 1 for January, 2 for February, 3 for March, and so on* 

Locations CONl and C0N2 contain constants of 1 and 2, respectively. 
No other constan t s are available . Use HOLD as temporary storage for TUBES. 

Use both the TR2 and TRN conunands in writing this program. Write 
your preliminary description at the left first, paying special attention 
to the data processing portions. 

Write the complete program below: 



COMPUT 
LAST 



MONTH 1 



M0NTH2 



STOP 



ERLC 



Turn to Page 52-1 for the correct answer. 



(Appendix -76} 



29A [in course] 



Problem 2 

Post Personnel wants a program that will count the number of men who 
have been in Alaska and the number who have been in Asia . The results 
are to be stored in ALASKA and ASIA. 

The words for each man are in memory starting at location TROOPS. The 
words are: serial number^ rank ^ dependent status^ assigned unit^ last over 
seas area^ and MOS* POST contains the number of personnel on Post. The 
overseas area is coded by number: 1 for Alaska, 2 for Asia, 3 for Europe, 
and so on . 

Locations Kl and K2 contain constants of I and 2, respectively. No 
other constants are available . Use SAVE as temporary storage for POST. 
Use both TRZ and TRN commands in writing this program. 

Write your preliminary description at the left first, paying special 
attention to the data processing portions. 

Write the complete program below: 



MODI FY 



DONE 



NORTH 



SOUTH 



STOP 



ERIC 



Turn to Page 30-2 for the correct answer. 
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Problem 5 

A business firm wants g program t hat will count the number of men 
who scored between 91 and lOD on a screening test> and the number who 
scored between 61 and 90 . the results are to be stored in EXCEL 
and GOOD. 

The words for each man are in tnemory starting at location TEST. 
The words are: titlc^ salary, dependent status, branch, test score, 
and health. MEN contains the number of men in the firm. Test scores 
are coded by number: 1 for 91-100, 2 for 81-90, 3 for 71-80, and so on. 

Locations ONE and TWO contain constants of 1 and 2, respectively. 
No other constants are available . Use KEEP as temporary storage for 
HEN. Write your preliminary description at the left first, paying 
special attention to the data processing portions. 

Write the complete program below: 



CHANGE 



OVER 



FIRST 



SECOND 



STOP 



Turn to Page 33-1 for the correct answer. 
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Answer to Problem 2, Page 29A 

Post Personnel wants a program that will count the number of men who 
have heen in Alaska and the number who have been in Asia, The results 
arc to be stored in ALASKA and ASIA. 

The words for each man are in memory starting at location TROOPS- The 
words are: serial number, rank, dependent status, assigned unit, last over 
seas area, and ^K)S. POST contains the number of personnel on Post. The 
overseas area is coded by number: 1 for Alaska, 2 for Asia, 3 for Europe, 
and so on* 

Locations Kl and K2 contain constants of 1 and 2^ respectively* No 
other constants are available . Use SAVE as temporary storage for POST. 
Use both the TR2 and TRN commands in writing this program* Write your 
preliminary description at the left first, paying special attention to the 
data processing portions* 

This is the program: 



MODIFY 



SOUTH 



STOP 



CLA 


POST 


TR2 


STOP 


STR 


SAVE 


CLA 


2R0 


STR 


ALASKA 


STR 


ASIA 


CLA 


TROOPS+4 


SUB 


K2 


TRN 


NORTH 


TR2 


SOUTH 


CLA 


SAVE 


SUB 


Kl 


STR 


SAVE 


TRZ 


STOP 


CLA 


MODIFY 


ADD 


K2 


ADD 


K2 


ADD 


K2 


STR 


MODIFY 


TRU 


MODIFY 


CLA 


ALASKA 


ADD 


Kl 


STR 


ALASKA 


TRU 


DONE 


CLA 


ASIA 


ADD 


Kl 


STR 


ASIA 


TRU 


DONE 


HLT 





Now continue your study with Page 29B. CAppendix-79) 
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Answer to Pro blem 1> Page 29 

The Base Electronics Warehouse wants a program that will count the num- 
ber of tubes used in January of last year, and the number used in February. 
The results are to be stored JAN and 

The words for each Jube are in memory starting at location VACUUM. The 
words are: part number, date received, shipment number, manufacturer, date 
tube was used^ and place tube was used* TUBES contains the number of vacuum 
tubes in the warehouse* The month in which the tube was used is coded by 
number: 1 for January, 2 for February^ 3 for March^ and so on. 

Locations CONl and C0N2 contain constants of 1 and 2, respectively* 
No other constants are available * Use HOLD as temporary storage for TUBES. 
Use both the TRZ and TRN commands in writing this program* 

Write your preliminary description at the left first, paying special 
attention to the data processing portions. 



This is the program : 



CLA 


■T"t 'trii^f' 

TUBES 


TRZ 


STOP 


STR 


HOLD 


CLA 


ZRO 


STR 


JAN 


STR 


FEB 


CLA 


VACUUM+4 


SUB 


C0N2 


TRN 


MONTHl 


TRZ 


M0NTH2 


CLA 


HOLD 


SUB 


CONl 


STR 


HOLD 


TRZ 


STOP 


CLA 


COMPUT 


ADD 


CON 2 


ADD 


C0N2 


ADD 


CON 2 


STR 


COHPUT 


TRU 


COHPUT 


CLA 


JAN 


ADD 


CONl 


STR 


JAN 


TRU 


LAST 


CLA 


FEB 


ADD 


CONl 


STR 


, FEB 


TRU 


LAST 


HLT 





Now continue your study with Page 29A. (Appendix-SO) 
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Answer to Problem 3^ Page 29B 

A business firm wants a program that will count the number of men who 
scored between 91 and 100 on a screening test, and the number who scored 
between 81 and 90. The results are to be stored in EXCEL and GOOD. 

The words for each man are in memory starting at location TEST. The 
words are: title, salary, dependent status, branch, test score » and health. 
MEN contains the number of men in the firm* Test scores are coded by num- 
ber: 1 for 91-100, 2 for 81-90, 3 for 71-80^ and so on. 

Locations ONE and TWO contain constants of 1 and 2, respectively. No 
other constants are available . Use KEEP as temporary storage for MEN. 
Write your preliminary description at the left first, paying special atten- 
tion to the data processing portions. 



This is the program: 



CHANGE 



SECOND 



CLA 


MEN 


TRZ 


STOP 


STR 


KEEP 


CLA 


ZRO 


STR 


EXCEL 


STR 


GOOD 


CLA 


TEST+4 


SUB 


TWO 


tr;; 


FIRST 


TRZ 


SECOND 


CLA 


KEEP 


SUB 


ONE 


STR 


KEEP 


TRZ 


STOP 


CLA 


CHANGE 


ADD 


TWO 


ADD 


TWO 


ADD 


TWO 


STR 


CHANGE 


TRU 


CHANGE 


CLA 


EXCEL 


ADD 


ONE 


STR 


EXCEL 


TRU 


OVER 


CLA 


GOOD 


ADD 


ONE 


STR 


GOOD 


TRU 


OVER 


HLT 





Now continue your study after Page 29B. 
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Appendix B 
LEVEUOF^ASPIRATION SCALE 



Name 



Before you continue your instruction^ tell us what percentage you think 
you scored on the test you Just completed : Make an x on the line for your 
estimate. The numbers are only guides . For example, if you think you scored 
99%, your X would be close to the end of the line. 



0% 



5 

1 1 1 1 1 


)% 

1 


9 

1 1 




! I 1 1 1 1 1 1 1 1 



100% 



15% 



75% 



Now tell us what score you think you will make on the test in the next 
section; Remember, make an X wherever you think you will score* 



0% 



1 1 


L 1 


50% 
1 1 


1 1 


9( 


J 


1 1 

15% 


1 1 


I ! 


1 1 

75% 


1 


1 



1100% 
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Appendix C 



SAMPLES OF REVISED COURSE FORMAT 



The samples in this appendix illustrate the content and format used in the 
final revised vei^sion of the course. The first three pages are excerpts (non- 
continuous) from the co'irse, showing presentation of instructional content with 
six practice problems. The following two pages are excerpts from the separate 
answer booklet, showing the answers to the six problems. 

problem 1.1 . There are two kinds of vacuum tubes in an 2 

^inventory^ 6SN7 tubes and 6AQ6 tubes. Write a program J 

I to compute the total value of both kinds^ placing the j 

? answer in TOTAL. The number of 6SN7 tubes is in STOCKl. ^ 

J The number of 6AQ6 tubes is in ST0CK2. Temporary stor- | 

^ age will be TEMP and TEMP+1. The cost of a 6SN7 is in J 

I VALUE and the cost of a 6AQ6 is in VALUE+1. Store the I 

I total worth of 6SN7 tubes in VALSTK and the total worth ^ 

J of 6AQ6 tubes in VALSTK+1. Assume at least one tube of | 

I each type . There is a 1 in KON. If you have to^ use J 

k COMPUT as symbolic location for the first looping pro- I 

Jgram^ NEXT for the second^ and SUM to get the total of I 
J both. Use asterisks wherever possible . 

^ Problem 1.2 . This problem is like the preceding one » k 

^except that the number of each type of tube is not known. | 

I Instead t each tube has information on its type stored J 

5 relative to TUBE. Each location in the series has' either ^ 

^ a 1 or a 0 — a 1 to indicate a 6SN7 tube^ and a 0 to mean k 

ka 6AQ6 tube. The total number of all tubes is in STOCK^ | 

1 which can be saved in TEMP. Assume at least one tube . J 

2 Use NEXT for sortings LAST for the test for completion, ^ 
^ and COMPUT for computation outside the loop. Use aster - | 
^ isks wherevt^r possible . VALSTK and VALSTK+1 will not be | 
I needed. You can add VALUE into TOTAL in one subroutine J 
J^nd VALUE+1 into TOTAL in another. ^ 



Symbolic Locations with Address Arithmetic 

Tracing out a program that uses asterisks becomes a 
little difficulty especially for someone else^ when 
the skips are much greater than 9 instructions. In 
these situations, you are better off using 3 symbolic 
location with address arithmetic. 

TRU DCME+2 means, "Transfer to the instruction two 
steps after symbolic location DONE." 

What instruction would be performed after TRU OK-1 below? 

CLA THIS 
OK SIR THERE 
TRU OK-1 



ERLC 



57 



Review : '[here i^rc throo ways to trans fcr to instruc- 
tion or tio refer to It 'for address Kiodif icat ion : {1) by 
usin^' its syinbo 1 i;;- locitt i.on , as in the? past ; (2) by ii.siilg 
an as te r i sk an J the numhor -of its pos i t ion in ro;l at ion 
to the instruction containing, the as4ft?risk ; and (7>) by 
using a symbolic location and ^t ho number of the instruc- 
t ion's positi(jn in relation to that symbol ie locat i on - 
For example ; . 

LOOP c:la Tills c:la this v cla tut^s 

STR TiJAT STR TilAT l.OOP -\STR THAT 

TRU LOOP . . TRIJ *-2 ' TRII LOUP-1 



Use as few symbolic locations as possible in the fol- 
lowing problems by (I) using the * for transfer of 9 
steps or lessj and (2) address arithmetic relative to 
a symbolic location for transfers of more steps. 

J Problem 1.4 . There is information about the ages and 
I ranks of enlisted men and officers starting at loca- 
I tion INFOj in the following three-word format: enlisted 
'"man or officer, age, and ranlt . The first word is coded 
^ with a + to indicate an officer, or a - to indicate an 
I enlisted personnel. Count the number of enlisted per- 
k sonnel in location TOTEM. The total number of personnel 
I is in location MBN. A constant of 1 is in Kl and a con- 
J stant of 3 is in K3. Use EM as symbolic location for 
I counting. Use DATA to modify the record to be processed. 
(The + a.n^J - signs can be considered the same as posi- 
tive and negative numbers.) 



^ Problem 1.5 . Headquarters wants to Itnow the niimber of 
I Second Lieutenants eligible for promotion to First 
? Lieutenant at the end of this month. Eighteen months 
!l active duty in grade are required for promotion from 



H Second Lieutenant to First Lieutenant, The data for 

officers are stored in 5-word records as follows: rank^ 
L serial number^ months in grade on active duty^ MOS^ and 
I assigned unit. Ranks for officers are coded by 1 for 
I Second Lieutenant> 2 for First Lieutenant^ and so on. 
i The records are kept relative to RATING. PERSON contain.i 
J the number of personnel. This number is not to be 
J destroyed. Store it temporarily in HOLD. Assume at 
^ least one officer. An 18 is in TIME^ a 1 is in KON^ and 
I a 5 is in K5. Store the number of eligible officers in 
I UP, Use RANK to modify addresses referring to rank> 
J LAST for the completion test, and SECOND for instructions 
\ referring to time in grade if the officer is a Second 
lieutenant. 



In effect, ADD DATA,IR2 says to copy DATA into a 
special temporary location CTR2) ancl there add to it 
the contents of that location [IR2). As IR2 contains 
the number I, DATA^l results. TIius, the complete 
instruction says the snine thln^ ;is: ADD DATA+l. 



I Pi;obJ^ciTi_2^. WritL* an InUtruction that says the same J 
J tliijitj as SMf^ COS'l + l. hulcx Register 3 contains the ^ 
j^minihe^j^^ . ^^^^^ ^ ^ ^ m ■ ^^J^ 



^ Problem 2.2 . iVritc an Ln^truction tliat says the same J 
1 tliijii; us STIi fiOOK+2. He^isLer I cojitains tJ)c ^ 

number 2. ^ 



'i'he Comma 

IVhy the comma? 

It is needed to separate two address fields^ the 
one that we have been using all along and a new one 
that is used in indexing. 

Up to nov^j you have been working with symbolic loca- 
tion fields > operational codes [the commands), and a 
single address field. For example: 

Symbolic Location Op. Code Address Field 

REPEAT ADD COST 

The address field you have been using is called the 
Firsj Address Field, All addresses that we have used 
previously appeared in the First Address Field, 

Indexing uses another address fields known as the 
Second Address Field, And, thus* the comma. Anytime 
more than one address field is used, a comma is required 
to keep them separated. Hence » the comma; 

ADD DATA J I R2 

What is the address field for DATA in the instruction 
above? IR2? 



ANSWERS TO PROBLEMS OF PHASE III: DATA PROCESSING 



Problem 1.1. 



CLA REPEAT 
ADD ONE 
STR REPEAT 



Problem 1.2. 



CLA HUBERT 
ADD DIGIT 
STR HUBERT 



Problem 1.4. 



CLA 2R0 
STR FINAL 



REPEAT CLA VALUE 

ADD FINAL 

STR FINAL 

CLA REPEAT 

ADD K 

STR REPEAT 



Change VALUE to VALUE+1 so that on 
the next loop VAlUE+1 will be added 
into FINAL. 



TRU REPEAT 



Problem 1.5. 

CLA ZRO 
STR EVERY 

A(iAIN CLA COST 
ADD EVERY 
STR EVERY 

CLA AGAIN COST is modified to COST+2 for the 

ADD TWD next loop. 

STR AGAIN 



TRU AGAIN 



Problert 2.1 . 



CHECK 



TEST 



STOP 



riA 


7Rn 


o 1 r\ 


DUMUO 


CLA 


PUSH 


TRZ 


J LO 1 




DUNUO 


ADD 


UNIT 


STR 


BONUS 


CLA 


MFN 


SUB 


UNIT 


STR 


MEN 


TRZ 


STDP 


CLA 




ADD 


UNIT 


STR 


CHECK 


TRU 


CHECK 


HLT 





Copy in a number. 

If it's a zertit skip to the test for 
completion. 

If it's not zero, count the salesman 
here; he must have made at least one 
sale. 



Test for completion. 



Address modification. 



Loopi ng. 



Problem 


2.2. 






CLA 


MEDICS 




STR 


TEMP 




CLA 


ZRO 




STR 


CALLS 


AGAIN 


CLA 


DOCTOR 




TRZ 


OUT 




CLA 


CALLS 




ADD 


ONE 




STR 


CALLS 


DUT 


CLA 


TEMP 




SUB 


ONE 




STR 


TEMP 




TRZ 


STOP 




CLA 


AGAIN 




ADD 


ONE 




STR 


AGAIN 




TRU 


AGAIN 


STOP 


HLT 





AGAIN 



OUT 



CLA MEDICS 
STR TEMP 

CLA ZRD 
STR CALLS 



CLA DOCTOR 
TRZ OUT 



if zero 



OA TEMP 
SUB ONE 
STR TEMP 
TRZ STOP 

I 



STOP 



CLA AGAIN 
ADD ONE 
STR AGAIN 

I TRU ^GAIN I 

HLT 



if not zero 



1 



CLA CALLS 
ADD ONE 
STR CALLS 
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In continuing research into the technology of training, a study was undertaken 
to devise guidelines for applying programed instruction to training courses 
that involve the learning of principles and rules for use in problem solving. 
As the research vehicles a portion of the material in the Army^s ADPS Programing 
Specialist Course was programed to explore several different factors in using 
automated instruction to teach computer programming. Experimental versions of 
the course were administered to over 900 subjects in various experimental group- 
ings- Criterion and retention tests based on actual job problems were used to 
measure subjects* performancej along with in-training measures* Results in a 
series of prompting/confirmation variations indicated that giving subjects 
extensive stimulus support during training helps motivate them and improves 
scores during training j but hampers them in using what they have learned* 
Requiring subjects to fully write out rules during training hindered them in 
developing problem* solving skills applying these rules; however, using mnexnonics 
(writing only the names of the rules) during training aided subjects in retaining 
what they had learnedj particularly for more complex material. Working with a 
variety of practice problems facilitated the learning of problem-solving skills. 
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